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100  SUMMRy 

101  Purpose 

The  New  York  University  Human  Engineering  Project  is  con 
cerned  with  the  evaluation,  from  the  human  engineering  point 
of  view,  of  an  airborne  Combat  Information  Center  (Cadillac 
III) »  The  purpose  of  this  report  is  to  describe  the  nature 
of  the  research  problem  by  listing  the  variables  which  pos¬ 
sibly  influence  the  system's  performance,  to  state  some 
principles  for  systems  research  which  have  been  evolved,  and 
to  report  certain  tentative  resultso 

1,2  Source  of  Information 


The  data  reported  were  gathered  in  some  50  orientation 
runs  of  the  system.  The  remaining  portions  of  the  report 
are  derived  from  that  data,  from  laboratory  experience,  and 
from  extensive  staff  conferences, 

1,3  The  Structure  of  the  Problem 

As  the  result  of  an  Intimate  consideration  of  the  research 
problem,  it  has  been  possible  to  prepare  a  series  of  statements 
which  provide  a  preliminary  structure  for  further  study.  The 
following  topics,  in  particular,  which  are  discussed  in  the 
body  of  the  report,  will  Illustrate  the  nature  of  the  structure 
evolved , 
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1«4  Recommendations 

It  is  recommended  that  the  statement  of  the  research 
problem  in  its  various  aspects  be  carefully  examined  by  cogni¬ 
zant  agencies 0  Suggestions  should  be  conveyed  to  the  NYU 
group,  through  the  Special  Devices  Center  Human  Engineering 
Division,  to  ensure  that  the  research  results  are  appropriate 
to  the  task. 
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2,0  INTRODUCTION 


During  the  course  of  the  development  of  the  airborne  Combat 
Information  Center,  the  Bureau  of  Aeronautics  recognized  the 
need  for  Investigating  how  the  equipment  in  that  Center  could 
be  used  most  efficiently  by  the  crew.  As  a  consequence,  the 
Special  Devices  Center,  Office  of  Naval  Research,  was  asked 
to  contract  for  a  human  engineering  study  of  the  airborne  Combat 
Information  Center t  The  New  York  University  College  of  Engineer¬ 
ing  was  selected  to  do  this  work* 

This  report  will  discuss  the  problem  of  studying  an  airborne 
CIC  from  a  human  engineering  viewpoint.  Before  an  outline  of  a 
research  program  is  presented,  consideration  will  be  given  to 
what  an  airborne  CIC  is  and  what  it  is  supposed  to  do,  what 
human  engineering  is  and  how  it  can  Improve  the  performance  of 
a  system,  and  what  research  methods  are  appropriate  for  this 
sort  of  study, 

2ol  The  Airborne  Combat  Information  Center  and  What  It  Does 
The  Combat  Information  Center  (CIC)  is  a  space  in  a  ship 
or  aircraft  so  equipped  and  manned,  so  arranged  and  organized, 
as  to  provide  for  the  collection,  display,  evaluation  and  dis¬ 
semination  of  combat  information.  It  performs  the  combat 
control  functions  assigned  to  it. 

Information  is  received  from  various  sources  such  as  radar, 
intelligence  reports,  radio,  radio  direction  finders  (RDF), 
electronic  counter-measure  equipment  (ECM) ,  sonar,  visual  look¬ 
outs,  aerological  data,  operation  orders,  and  other  publications* 
This  information  must  be  sorted  and  displayed.  It  must  then 
be  analyzed  as  to  its  tactical  meaning.  That  requiring  action 
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must  be  acted  upon;  that  requiring  dissemination  must  be  relayed 
to  the  necessary  placeso 

During  the  latter  days  of  World  War  II,  low-flying 
Japanese  aircraft  were  able  to  approach  within  20  to  25  miles 
of  our  Naval  forces  without  being  detected  by  shlpborne  radar. 

To  correct  this  limiting  of  coverage  resulting  from  radar's 
line-of-slght  characteristics,  the  radar  set  Itself  was  sent 
aloft,  and  then  more  and  more  auxiliary  equipment  was  included. 
Each  succeeding  version  of  the  airborne  CIC  has  been  developed 
to  provide  more  potentiality.* 

The  particular  version  of  the  airborne  CIC  under  study  is 
the  third--Cadillac  III— that  system  Installed  in  the  Lockheed 
Constellation  aircraft  designated  by  the  Navy  as  the  PO-IW. 

It  should  be  noted  that  the  PO-IW  is  an  experimental  model. 

The  airframe  chosen  for  the  operational  Airborne  CIC  aircraft 
is  the  larger  version  of  the  Lockheed,  Navy  designation  P0-2W, 
called  commercially  the  Super-Constellation.  Hereafter  all 
references  will  be  to  the  operational  P0-2W. 

Certain  assumptions  regarding  the  operational  use  of  the 
P0-2W  have  been  found  to  be  appropriate  and  can  be  stated  as 
follows* 

2ol.l  Task  Group  Oreanlzatlont  The  nature  of  the  tactical 
situation  facing  the  combat  control  system,  of  which  the  P0-2W 
is  a  part,  requires  centralized  control  with  decentralized 
action.  This  would  mean  that  any  air  control  functions  per¬ 
formed  by  the  airborne  CIC  would  have  to  be  explicitly  authorized 

♦For  a  more  detailed  discussion  of  the  history  and  development 
of  the  airborne  CIC,  see  Technical  Report  SDC  279-3-5,  "Layout 
of  the  Combat  Information  Center  in  the  P0-2W  Aircraft." 
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either  by  the  provisions  of  the  Operation  Order  or  by  the 
Officer  in  Tactical  Conmand  (OTC) ,  who  usually  commands 
the  Task  Group  with  which  the  P0-2W  operates,  and  whose 
staff  prepares  the  Operation  Order.  The  principle  of  central¬ 
ized  control  requires  that  the  reports  made  by  the  airborne 
CIC  go  to  the  OTC,  to  such  places  as  he  directs,  or  where 
the  Operation  Order  specifies* 

2.1.2  FUnctiont  The  primary  function  of  the  P0-2W,  as 
defined  by  the  Chief  of  Naval  Operations,  is  that  of  airborne 
early  warning  (AEI?).  Its  secondary  function  is  that  of  con¬ 
trolling  air  intercepts.  Additional  functions  may  include* 
anti-submarine  warfare  (ASW),  air  control,  air-sea  rescue 
coordination,  weather  reconnaissance,  and  amphibious  landing 
coordination.  In  more  detail,  these  functions  are* 

1)  Detection.  Search,  detection,  and  tracking  of 
targets  within  an  area  not  covered  by  the  radar 
of  surface  ships. 

2)  Evaluation.  The  evaluation  of  targets  as  to 
friend  or  foe,  air  or  surface,  raid  or  snooper, 
number  and  type  of  enemy  weapons  in  each  raid. 

3)  Air  control.  Offensively,  this  would  Include 
such  duties  as  providing  friendly  bombers  with 
warning  of  the  approach  of  enemy  fighter  groups, 
of  acting  as  navigational  director  to  friendly 
strike  forces,  of  directing  mine  laying  aircraft 

to  the  point  of  release  of  their  mines.  Defensively, 
it  would  Include  control  of  on-station  CAP  in"  the 
interception  of  enemy  snoopers,  vectoring  of  killer 
aircraft  to  submarine  contacts,  elding  returning 
atrlkes  and  interceptors  to  effect  rendezvous 
with  their  home  base,  and  performing  ”de-lousing*‘ 
actions. 

4)  Reporting.  Giving  evaluations  of  tactical  situ¬ 
ation  to  the  OTC,  reporting  detailed  positions, 
courses,  speeds,  and  altitudes  of  enemy  raids  to 
other  CICs  into  whose  area  raids  are  going. 
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2olo3  Operating  Area:  In  performing  its  airborne  early 
warning  and  "anti-snooper”  functions,  the  P0-2W  will  have  to 
operate  at  that  distance  from  the  force  which  will  provide 
adequate  early  warning  and/or  snooper  protectiono* 

2.1o4  Comm.unicatlon  Linlrs:  The  primary  communication  links 
at  such  a  distance  from  the  force  will  be  MRP  or  CW  radio,  or 
VHF  or  UHF  radio  through  relay  statlonso  At  this  distance, 
and  at  the  present  stage  of  technical  development,  the  use 
of  the  PO  link  to  OTC  seems  unlikely.  There  very  well  may  be 
several  communication  nets:  one  which  links  the  various  P0-2Ws 
to  the  OTC,  and  others  which  link  the  P0-2Ws  to  other  CICs* 

The  amoimt  of  traffic  will  crowd  these  nets  so  that  careful 
consideration  will  have  to  be  given  to  the  type  and  amount  of 
information  permitted  on  the  channel. 

2.1.5  Command  of  the  Airborne  CIC  Aircraft:  For  the  airborne 
CIC  to  accomplish  its  stated  mission  most  effectively,  the 
CIC  Officer  must  be  in  command  of  the  plane  while  it  is  on 
station.  If  the  CIC  Officer  deems  it  necessary  to  change  the 
flight  pattern  of  the  plane  because  of  the  tactical  situation, 
his  decision  must  govern. 

2.1.6  Load  Conditions:  The  load  condition  to  be  faced  by  the 

Combat  Control  System  is,  of  course,  most  difficult  to  predict. 

The  load  condition  for  a  single  picket  such  as  the  airborne  CIC 

may  vary  even  more.,.  An  assumption  is  made  that  it  may  range 

♦An  operational  research  study  on  the  function  of  the  airborne 
CIC  is  in  progress  under  the  direction  of  the  Special  Devices 
Center.  The  published  report  of  this  study  will  indicate 
the  ideal  operating  conditions  for  this  system. 
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between  20  and  $0  raids  per  hour. 

2.2  Research  Task  As  Stated  In  the  Contract 

New  York  University  has  been  instructed  to  conduct  research 
(xmder  the  cognizance  of  the  scientific  officer  assigned  by  the 
government)  in  the  following  areass 

(a)  Conduct  a  study  of  existing  and  proposed  airborne 
CIC  installations  for  the  purpose  of  (1)  ascertaining  the 
most  efficient  layout  of  equipment  and  controls  for  CIC 
functions,  and  (2)  I’uvestigatlng  the  possibility  of  adding, 
eliminating,  or  modifying  component  equipment  of  the  CIC 
system  ^or  more  efficient  operation. 

(b)  Investigate  new  techniques  in  presentation,  layout, 
and  operating  procedures  with  particular  emphasis  on 
reductlpn  of  operator  fatigue. 

(c)  Hake  recommendations  to  the  Bureau  of  Aeronautics 
via  the  Special  Devices  Center  of  the  Office  of  Naval 
Research  for  desirable  changes  in  CIC  equipment  which  can 
be  incorporated  in  models  now  or  soon  to  be  developed. 

(d)  Collaborate  with  other  research  and  development 
groups  to  assist  in  the  submission  of  a  study  of  ai.  ideal 
airborne  CIC  system  which  is  considered  capable  of  develop¬ 
ment  in  the  next  five  years. 

There  were  a  number  of  reasons  why  this  task  could  not  be 
stated  in  more  detail.  The  rapid  rate  at  which  new  electronic 
devices  are  being  developed  has  many  tactical  implications  for 
the  military  organizations,  of  which  the  airborne  CIC  is  only  a 
part.  Thus  there  must  be  constant  effort  on  the  part  of  each 
element  of  the  larger  program  to  remain  flexible  to  new 
developments;  ultimately  the  parts  must  fit  together.  An  ap¬ 
preciation  of  this  situation  keeps  the  contracting  agency  from 
restricting  the  range  of  the  research  work. 

Equally  important  has  been  the  fact  that  because  human 
engineering  systems  ^tudy  is  relatively  new,  the  precise  nature 
of  the  results  of  such  study  could  not  ue  anticipated.  The 
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contract  terms  must  permit  the  exploration  of  both  subject 
matter  and  techniques;  the  research  task  will  be  defined  in 
part  by  the  effectiveness  of  the  techniques  developed  by  the 
researchers. 

Broadly  Interpreted,  the  research  goals  are  two-fold i 
a)  research  leading  to  the  best  utilization  of  present 
equipment  (that  designed  for  inclusion  in  the  P0-2W  Lockheed 
Constellation),  and  b)  research  leading  to  the  development 
of  future  versions  of  the  airborne  CICo  The  PO-IW,  and  its 
associated  electronic  gear,  was  in+ ended  to  be  an  experimental 
airborne  CIC  to  be  tested  operationally  and  experimentally. 

The  world  situation,  however,  made  the  need  for  a  production 
model  of  the  airborne  CIC  more  urgent;  it  has  been  necessary 
to  proceed  with  that  design  with  limited  experience. 

2,2ol  The  Wav  the  Contract  Will  be  Carried  Out>  Inferences  as 
to  the  kinds  of  results  that  may  be  expected  from  this  project 
may  be  drawn  from  the  following  assumptions  regarding  the  relation 
between  New  York  University  and  organized  agencies* 

a)  In  improving  the  performance  of  the  CIC  system, 

New  York  University  is  to  exploit  the  capacities  of  the  human 
being  by  so  designing  equipment  and  procedures  that  operation  is 
adapted  to  man’s  abilitieso  Recommendations  for  Improvement  of 
the  system  will  be  made  in  this  order  of  preference* 

1)  Statement  of  preferable  operating  procedures 

li)  Adaptation  and  modification  of  present  gear, 

or  gear  being  developed  currently,  with  respect 
to  kind  and  positioning  of  controls,  design  of 
displays  and  dials,  size  and  shape  of  component, 
and  lighting  facllltleso 
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111)  Suggested  development  of  displays  and  operator 
aids  to  facilitate  performance* 

Iv)  The  addition  or  deletion  of  machine  and/or 

operator  links*  The  addition  of  a  machine  link 
will  he  suggested  only  If  It  Is  to  be  a  simple 
electro-mechanical  device* 

v)  Suggested  develop’^ent  of  new  electronic  equipment 
only  when  other  methods  will  not  suffice* 

b)  In  comparing  alternative  evaluations  of  equipment, 
doctrine  or  layout,  the  present  equipment,  existing  doctrine  and 
the  present  layout  must  be  cne  of  the  arrangements  that  Is  tested* 

c)  It  Is  In  the  best  interest  of  the  Navy  that  New  York 
University  not  limit  Its  concept  of  operating  procedures  and 
doctrine  to  present-day  operational  thinking  in  proposing  al¬ 
ternative  methods  of  operation*  NYU  must  temper  Its  suggestions, 
however,  by  thorough  discussion  of  i^s  Ideas  with  operating 
personnel, 

d)  While  there  are  many  '  ays  of  deriving  recommendations 
to  improve  the  system,  the  project  Is  ll'-ilted  to  those  sug¬ 
gestions  which  It  can  support  by  fact* 

e)  In  general  NYU  conceives  Its  role  as  a  consultant  as 
one  In  which  It  Is  obliged  to  describe  In  explicit  terms  the  load¬ 
carrying  capacity  of  the  airborne  CIC  under  various  conditions*  For 
example,  New  York  University  will  be  able  to  describe  the  load¬ 
carrying  capacity  of  the  system  when  different  numbers  of  consoles 
are  used.  The  Navy  can  then  determine  on  the  basis  of  tactical 
need  how  many  consoles  to  provide  in  an  airborne  CIC.  The  neces¬ 
sary  facts  are  thus  available  to  determine  requirement,  com¬ 
promising  the  ideal  with  the  practical. 
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f)  The  NYU  Human  Engineering  Project  is  obliged  to 
draw  on  basic  data  which  are  contained  in  the  professional 
literature  of  the  various  disciplines  which  it  represents  and 
in  classified  research  reports  of  other  government  agencleso 
Coordination  w’lth  the  Special  Devices  Center  Human  Engineering 
Division  and  its  contractors  may  result  in  reallocation  of 
research  tasks  which  are  more  appropriate  to  the  respective 
contractor's  facllitiesn  Fundamental  research  will  be  performed 
only  when  essential  to  the  progress  of  the  project  and  when 
information  is  not  otherwise  availableu 

g)  In  order  that  the  value  of  experimental  results 
obtained  on  this  project  may  be  fully  realized,  NYU  should 
consult  with  cognizant  agencies  during  planning,  design  and 
construction  phases  of  the  development  programo 

2o3  NYU  Hunan  Engineering  Project  Research  Facilities 

This  section  will  describe  the  equipment  now  contained  in 
the  NYU  Human  Engineering  Project  laboratoryo  How  completely 
the  airborne  CIC  has  been  duplicated,  the  extent  of  the  Inputs 
to  the  system  simulated,  and  the  facilities  for  observing  systems 
performance  will  be  illustrated. 

2o3ol  The  Airborne  CIC  Itself;  Figure  1  pictures  the  laboratory 
airborne  CICo  The  following  components  of  the  airborne  CIC 
have  been  Installed: 

a)  One  set  of  the  APA-56  (XN-1)  equipment  which  consists 
of  five  PPI  repeater  consoles  with  associated  equipment  —  video 
insertion,  recording  camera,  electronic  grid  map  insertion,  and 
ground  stabllizationo* 

♦Ground  stabilization  is  presently  being  accomplished  by  a  slmu- 
lator,  the  APA-57  equipment  having  been  removed  for  another 
BuAer  projecto 
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Flpure  1  -  The  CIC  portion  of  the  Lockheed  P0-2W  simulated  In 
the  NYU  Human  Engineering  Project  laboratory  at  the  Special 
Devices  Center. 
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b)  A  simulated  APA-56  height  finder  console  made  by 
General  Precision  Laboratories  including  the  priority  and  data 
transmission  systems* 

c)  A  Lockheed  intercommunications  system  which  includes 
both  that  for  the  CIC  and  flight  crew  —  only  part  of  the  flight 
crew  facilities  are  being  used. 

d)  Simulation  of  the  command,  liaison,  VHF  radio  trans¬ 
ceivers.  Four  VHF  channels  are  provided. 

e)  The  two  P0-2W  edge-lighted  status  boards  have  been 
duplicated  and  mounted.  In  addition,  a  visual  aids  projector 
developed  by  the  Special  Devices  Center  for  other  purposes  is 
being  used  as  one  experimental  display. 

f)  This  equipment  has  been  contained  in  a  hull  section 
on  which  blowers  have  been  put  in  an  effort  to  duplicate  the 
air  conditioning  system  of  the  plane.  The  lighting  in  the  P0-2W 
has  been  duplicated,  additional  facilities  have  been  provided, 
hov.’ever,  so  that  this  factor  can  be  varied  for  experimental 
purposes.  An  audio  sound  generator  has  been  installed  to  add 
the  realism  of  engine  noise. 

2.3o2  The  ”Radar"  Inputs  In  order  that  there  would  be  a  readily 
controlled  "radar"  input  to  the  repeater  scopes,  the  Special 
Devices  Center  contracted  with  the  General  Precision  Laboratory 
for  the  construction  of  a  target  simulator.  This  sjmulator  pro¬ 
vides  the  relative  movement  of  24  air,  5  surface,  and  one 
stationary  target  wjtn  respect  to  the  plane.  (The  ground 
stabilization  equipment  aboard  the  plane  must  then  translate 
this  relative  movement  into  a  ground-stabilized  picture.) 

Each  of  the  targets  can  be  initially  positioned  at  any  point 


on  a  480  mile  square  grid,  can  be  keyed  to  respond  or  not  respond 
to  IFF,  can  take  any  altitude  from  0  to  57»000  feet,  and  can  be 
varied  as  to  brightness.  The  r^oving  targets,  and  the  PO-27’ 

Itself,  can  be  given  any  course;  air  targets,  and  the  P0-2W, 
can  be  ranged  in  speed  from  100  to  600  knots,  the  surface 
targets  from  0  to  40  knots.  From  20  to  120  knots  of  wind  can 
be  applied  to  the  air  targets  as  a  whole.  Sea  return  is  also 
simulated  and  the  radar  picture  can  be  "smeared”  by  the  turns 
of  the  P0-2W.  Figure  2  sho^'-s  the  simulator  area  manned  by  the 
experimenters. 

2.3.3  The  Officer  in  Tactical  Command  (PTC)  g  The  Officer  in 
Tactical  Command  is  the  person  to  whom  the  output  of  the  airborne 
CIC  is  directed.  A  station  has  been  established  in  the  laboratory 
to  represent  this  element.  It  has  the  radio  communication 
facilities  that  would  be  nrovided  between  the  airborne  CIC  and 
the  OTCo  In  addition,  it  has  a  plotting  surface  for  displaying 
the  information  that  comes  to  the  OTC. 

Actually  this  is  an  anomalous  position  because  it  is 
manned  by  experimenters  both  for  OTC  and  experimental  control 
purposes.  For  example,  the  plotting  surface  is  a  VG  scope  on 
which  the  radar  input  is  presented.  The  OTC  would  not  have 
such  a  picture  under  operating  conditions,  but  that  picture  is 
useful  in  the  laboratory  for  purposes  of  controlling  the  ex¬ 
periment  being  run. 

2.3o4  The  Experimenters'  Toolss  The  following  facilities  for 
observing  the  systems'  performance  have  been  or  soon  will  be 
installed  ? 
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Figure  2  -  The  experinenters’  portion  of  the  NYU  Human 
Engineering  Project  laboratory  showing  both  the  simulator 
and  some  of  the  experimenters*  toolso 
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a)  Six  tape  recorders  with  which  any  verbal  communication 
of  the  crew  can  be  recorded o  Other  signals  can  be  imposed  on 
this  verbal  record  for  measurement  purposes. 

b)  A  motor-driven  motion  picture  camera  and  special 
projector  for  micromotion  analysis  of  the  crew's  use  of  equip¬ 
ment. 

c)  A  VG  and  a  VJ  radar  scope  for  experimenters’  use  in 
observing  and  controlling  the  radar  input. 

d)  The  recording  camera,  which  is  a  part  of  the  APA-5^6 
equipment,  used  by  the  experimenters  in  some  cases  to  record 
the  radar  picture. 

e)  Special  communications  facilities  so  that  experimenters 
stationed  in  and  out  of  the  CIC  can  coordinate  their  efforts. 

2.4  Personnel  Assigned  to  the  Project 

To  complete  the  system,  the  Navy  has  assigned  a  crew  to  the 
project.  The  minimum  crew  necessary  for  an  operational  airborne 
CIC  would  be  as  follows:  a  CIC  Officer,  an  Assistant  CIC  Officer, 
four  Air  Control  Officers,  a  Navigator,  a  DRT  Operator,  an  Altitude 
Determining  Radar  Operator,  a  Search  Radar  Operator,  a  Radio 
Operator,  an  Electronics  Counter-Measures  Operator,  an  Electronics 
Technician,  and  a  Talker.  This  makes  a  total  of  fourteen,  not 
including  the  Pilot,  Co-pilot,  and  Flight  Engineer.  It  should  be 
emphasized  that  this  number  does  not  allow  for  the  necessary 
relief  of  personnel  during  long  operational  flights  under  heavy 
load.  Six  experienced  CIC  officers  serve  the  project  as  con¬ 
sultants  and  as  crew  members;  they  fill  the  first  four  of  the 
tabulated  positions.  Three  enlisted  men  at  present  represent 
the  remainder  of  the  crew.  Additional  enlisted  me.,  have  been 
requested . 
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New  York  University  has  assigned  full-ti^ne  industrial  and 
electronic  engineers,  experimental  psychologists,  technical  and 
clerical  personnel  to  the  project,  and  utilizes  a  number  of 
part-time  specialists  when  necessary,  making  a  total  of  approxi¬ 
mately  25  contractor  personnel. 
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3.0  THE  MEANnJG  OF  HUM/VN  ENGINEERING 

After  consideration  of  the  airborne  CIC  and  what  it  is 
supposed  to  do,  as  well  as  the  contract  terms  and  project 
facilities,  the  ouestion  logically  arises:  What  is  human 
engineering,  and  how  can  research  in  that  direction  improve  the 
performance  of  the  airborne  CIC? 

Since  the  beginning  of  time  man  has  been  devising  tools 
to  aid  him  in  obtaining  food,  shelter,  and  security.  Through 
the  years  he  has  harnesstd  the  energy  to  power  those  tools  and 
now  only  controls  them.  Where  the  man  of  the  Stone  Age  relied 
on  his  own  strength  to  shape  wood  to  his  needs,  modern  man  only 
guides  the  electric  sawo 

As  tools  are  built  that  perform  more  intricate  functions, 
the  job  of  controlling  them  gets  toughero  Guiding  an  electric 
saw  through  a  piece  of  wood,  driving  a  Model  T  do7m  a  country  road, 
and  flying  a  jet  fighter  plane  are  control  functions  of  increasing 
difficulty.  Nine  times  out  of  ten  the  niece  of  wood  can  be  cut 
properly;  perhaps  in  only  one  out  of  ten  "passes"  can  a  jet  plane 
shoot  down  an  enemyo  More  often  than  not,  the  failure  can  be 
attributed  to  "if^proper  controlling.,"  Thus,  as  mechanisms  are 
built  to  do  more  complicated  tasks,  Inadequate  f'ontrolling  limits 
the  effectiveness  of  those  mechanismso* 


♦One  way,  of  course,  to  do  away  with  human  error  is  to  do  away  with 
the  man.  This  attitude  is  expressed  in  today's  move  towards  "push¬ 
button  warfare,"  It  is  true  that  eleetro-mechanlcal  and  electronic 
devices  have  been  built  which  perform  marvels  in  controlling,  often 
outdoing  the  hu^^an  on  comparable  tasks.  These  devices  can  only 
perform  the  function  for  which  they  v'ere  designed,  while  man,  on 
the  other  hand,  spontaneously  performs  control  funi^tions  that  have 
not  been  anticipated.  Present  day  planners  must  not  lose  sight 
of  the  capable,  compact,  reliable  electro-mechanical  gadget 

of  all--man  hlmsel'f',  whose  full  capacities  have  not  yet  been  de- 
terninedc  Put  it  is  necessary  to  give  man,  the  controller,  a  break- 
make  it  easier  for  him  to  control  by  adapting  the  machine  to  human 
capacities. 
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3.1  The  Nature  of  Human  Control 

What  is  involved  in  the  human  control  of  machines?  In  general, 
good  control  is  making  the  appropriate  response  to  a  cue,  or 
stimulus,  at  the  proper  time  while  poor  control  is  either  making 
no  response  at  all,  making  an  erroneous  response,  or  being  too  slow 
or  too  fast  with  the  right  response* 

Good  control  requires  optimal  cue-response  associations.  These 
associations  will  depend  upon  three  things  wh:5ch  the  psychologist 
would  describe  as  follows:  physiological  limitations  in  man's 
sensory  and  motor  capacities;  motivation,  v;hlch  includes  the 
development,  or  reinforcement,  of  optimal  associations;  and  the 
basic  processes  of  discrimination  and  generall7ation  of  stimuli  and 
differentiation  and  induction  of  responses.  These  terms  can  best 
be  explained  by  illustrationo 

3.1.1  Physiological  limitations:  Limitations  in  human  sensory 
capacity  must  be  considered  in  providing  cues.  Man  cannot  hear 
frequencies  above  20,000  cycles  per  second,  for  Instance,  nor  can 
he  see  objects  unless  there  is  some  illumination.  Also  there  are 
limits  to  the  degree  to  which  man  can  tell  the  differences  between 
stimullo 

Man's  responses  are  restricted  both  by  his  anatomical 
measurements  and  by  the  strength  of  his  various  muscles.  The  fingers 
can  exert  less  force  than  the  shoulders,  for  instance,  and  there  are 
limits  for  both,  not  only  as  to  the  amount  of  force  which  can  be 
exerted  but  as  to  how  long  this  force  can  be  applied. 

3.1.2  Motivation:  Such  terms  as  ambition,  ’^orale,  anxiety,  con¬ 
scientiousness  describe  motivating  elements  of  human  behavior.  Even 
though  the  am.ount  of  motivation  may  be  difficult  to  measure,  the 
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degree  of  motivation  is  believed  to  affect  the  speed,  frequency, 
and  precision  of  responses.  When  a  particular  cue-response  as¬ 
sociation  satisfies  the  dominant  motivation,  one  can  say  the 
association  has  been  reinforced.  The  same  response  will  be  more 
likely  to  follow  this  cue  when  the  latter  occurs  again. 

For  exa’^ple,  a  soldier  may  make  a  variety  of  aiming 
responses  before  he  hits  the  target.  In  suV'senuent  atte^^nts,  the 
successful  aiming  response  will  occur  more  frequently.  The  ac¬ 
quisition,  improvement,  and  maintenance  of  this  response  will 
depend  in  part  on  the  soldier's  motivation — how  caper  he  is  to  hit 
the  target.  The^  will  also  depend  on  suoh  reinforce’^ent  factors 
as  how  often  the  response  is  successful  and  how  quickly  he  finds 
out  the  results.  Not  only  will  motivation  and  reinforcement  have 
a  big  hand  in  determining  how  oulckly  this  man  will  learn  good 
control  of  a  mechanism,  but  they  will  also  have  a  lot  to  do  with 
whether  such  control  continues. 

3.1.3  Discrimination  of  Stimuli  and  Differentiation  of  Responses; 
When  a  resp^^nse  is  learned  for  a  cue,  there  will  be  a  tendency-- 
perhaps  innate,  perhaps  learned — for  the  sane  response  to  occur  for 
"similar"  cues.  Suppose  an  operator  has  learned  to  throw  a  switch 
to  the  left  when  a  light  above  it  shows  red.  He  will  he  inclined 
■f'O  throw  a  second  switch  to  the  left  if  a  similar  light  should  show 
red.  Thjs  generalization  can  lead  to  unfortunate  errors.  If 
the  first  switch  should  be  turned  to  the  left  but  the  second  switch 
to  the  right,  the  tendency  to  make  the  same  left  response  might 
cause  an  accident. 

When  a  man  sets  a  control  knob,  there  will  always  be  some 
variation  in  the  ray  he  does  it  as  he  tries  to  find  a  response  that 
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is  successful--a  phenonenon  sometimes  called  -response  inductjon. 

On  the  other  hand,  the  process  of  iriakinp  responses  nore  stereotyped 
and  precise  once  he  has  made  a  successful  response  may  be  termed 
response  dif '"eren^lation.  It  is  the  prineipal  component  of 
muscular  skill,  together  v'ith  stimulus  dlscrlmlnationo  Like  the 
latter,  differentiation  comes  about  through  differential  reinforce¬ 
ment,  which  in  turn  depends  on  motlvatlone 

Such  are  the  things  which  determine  the  nature  of  human 
control  over  mechanisms.  This  discussion,  though  simplified,  does 
demonstrate  that  the  research  scientist  can  deal  with  man-machine 
rela rlonshlps  in  much  the  same  framework  that  psychologists  use  in 
describing  the  rest  of  human  behavior.  The  human  being  performs 
as  usual.  The  machine  supplies  the  cues  and  transmits  the  responses. 
Its  successful  operation  provides  the  reinforcement  responsible  for 
the  cue-response  associations  ’"hich  cor^prise  good  control# 

3.2  Human  Enrineering  Research 

There  are  two  distinct  aspects  to  human  engineering  research: 
the  study  of  components  and  the  study  of  systems.  A  component 
can  be  defined  as  a  man-machine  element;  the  term  system,  as  used  in 
this  report,  refers  to  a  particular  group  of  men  and  machines.  The 
group  in  question  is  a  team;  e:^fective  team  performance  depends, 
of  course,  not  only  upon  performance  of  the  components  of  that  team, 
but  also  upon  maintaining  the  proper  coordination  between  elements 
and  integrating  their  individual  efforts.  Human  engineering  research 
until  this  time  has  been  devoted  primarily  to  component  research; 
the  NYU  Human  Engineering  Project  is  one  of  the  first  experl’^ental 
research  projects  that  has  been  explicitly  directed  to  the  study 
of  a  system# 
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The  effe^'tive  perfornance  of  ei'*'h»  r  a  conponent  or  a  system 
depends  on  equipment  that  is  desipned  to  permit  the  develop^^ent 
of  optimal  cue-response  associations.  It  also  ^’epends  on  the 
selection  of  operators  who  can  operate  that  equipment,  and  on  train¬ 
ing  them  properly.  Training  Involves  the  establishment  of  motivation 
to  learn  and  the  reinforce’^ent  of  optimal  associations  through  reward 
and/or  punishment.  Whether  an  operator  continues  to  perform  ef- 
feetively  depends,  of  course,  on  maintaining  both  motivation  and 
reinforcement;  this  is  a  function  of  leadership. 

The  emphasis  in  human  engineering  research  is  upon  the  proper 
design  and  use  of  equipment.  The  selection  of  operators  is  a 
problem  for  another  branch  of  applied  psvchology,  as  is  the  es¬ 
tablishment  and  operation  of  training  progranis.  Maintaining  operator 
effectiveness  is  the  job  of  the  leader  in  the  operational  situation. 
3.2.1  Study  of  Components?  Tomponent  s^udy  concerns  itself  with 
the  exploration  of  roan’s  physiological  limitations  in  sensing  and 
responding.  It  also  deals  with  man's  capacity  to  discriminate 
stimuli  and  to  differentiate  responses.  The  type  of  control  problem 
illustrated  in  Section  3*1»?  by  the  example  of  the  switches  with 
similar  lights  is  common  in  co’^nonent  study  and  is  re  olved 
generally  by  modifi'^ing  the  equipment  or  the  operating  procedure. 
Training  in  discrimination  provides  ano-^-hcr  answer  to  the  same 
problem.  Redesign  is  preferable,  however,  because  discrimination 
training  leads  to  less  re’’iable  results.  For  cxa’^ple  sudden 
Increases  in  motivation  such  as  occur  in  panic  may  lead  to  respo’^ses 
which,  throup}]  discrimination  tral’^.lng,  had  become  subliminal 
under  conditions  of  normal  stress.  Thus,  in  a  sudden  emergency, 
even  the  highly-trained  operator  might  turn  the  switch  the  ^•’rong 


There  are  T^any  practical  considerat jor.s,  however,  in 
redesigninr  eauiprent  to  permit  the  development  of  optimal  cue- 
response  a ssociations 0  The  optimal  design  ''annot  always  be  adopted 
either  because  of  engineering  difficulties  or  because  soT^e  standard 
but  Inferior  design  has  already  been  widel;'’  adopted o  The  keys  on 
a  typewriter  can  be  arranged  in  a  more  efficient  way,  for  example; 
The  Dvorak  keyboard  is  better  than  the  standard  keyboard.  But 
millions  of  typists  would  have  to  be  retrained  to  the  new  keyboard, 
and  millions  of  typewriters  w.ould  have  to  be  modified. 

The  practical  result  of  human  engineering  study  of  components 
Improvement  of  equipment  or  procedure — can  be  done  only  through 
proper  regard  for  human  engineering  principles,  technical  design 
and  training  problems,  and  economic  factors.  The  hum.an  engineer 
can  fulfill  his  function  in  component  study  by  considering  these 
factors  and  participating  in  the  discussions  that  load  to  an  ap¬ 
propriate  compromlseo 

3.2.2  Study  of  systems?  The  study  of  a  system  involves  those  cor- 
sidorations  of  component  research  plus  additional  factors  pertaining 
to  group  activity.  How  the  equipment  is  arranged,  rhat  operating 
procedures  are  used,  what  communications  facilities  are  available, 
and  what  displays  are  provided  on  which  to  assemble  and  integrate 
information  are  all  critical  ouestions  with  which  system  study 
deals. 

System  study,  from  the  human  engineering  standpoint,  does 
not  concern  itself  with  ouestions  of  formal  training  any  more  than 
does  component  study.  However,  consideration  of  motivation  and 
reinforcement  fac''‘ors  may  become  more  critical  in  systems  research 
because  of  the  dynamic  Interactions  involved  In  group  behavior. 
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For  eyar^ple ,  thp  performance  of  opr-rators  gathering  information 
r^ay  he  seriously  affected  by  the  group  procedures  used.  One 
procedure,  by  facilitating  transmission  and  display  of  information, 
has  consequent  reinforcing  effer'ts  on  the  operators.  Another 
procedure  may  discourage  them  by  hampering  such  transmission  and 
display. 

For  the  experimenters,  an  appreciation  of  the  elements  that 
affeet  hu’^an  performance  leads  to  a  better  unde:^standlng  of  the 
way  a  system  operates.  Interpretation  of  results  may  be  more 
adeouate,  and  it  may  be  possible  to  develop  better  operating 
procedures. 

System  study  and  component  study  should  proceed  concurrently. 

The  design  of  a  mechanism  depends  on  the  function  reculred  of  the 
component  by  the  system;  on  the  other  hand,  a  system  design  depends 
on  the  full  realization  of  the  capacities  of  its  components. 

The  study  of  a  system  poses  real  difficulties  in  scientific 
methodology;  these  will  he  discussed  in  the  next  section  of  this 
report. 

3.3  The  Professions  Contributing  to  Human  Engineering  Research 

Frank  and  Lillian  Gilhreth  v/ere  among  the  first  hu’^oan  engineers; 
through  motion  analysis  thny  studied  the  operator’s  task  to  make 
it  simpler  for  him.  Since  then  industrial  engineers  and  methods 
engineers  have  improved  industrial  efficiency  with  Job  simplifications 
During  World  War  II,  the  experimental  psyehologist  began  to  make 
signiflcani  contributions  on  the  same  practical  value.  Phys-'.ologist s , 
physicians,  psychiatrists,  engineers  from  dl-f’:f’erent  areas  of 
speeializat ion ,  and  men  from  allied  fields  have  all  been  active 
in  this  area.  The  keynote  in  human  engineering  v,'ork  seems  to  be 
inter-disciplinary  cooperation. 
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It  is  hard  to  distinguish  between  activities  of  the  various 
professional  groups  jr.  terrs  of  subject  natter.  Each  of  these 
disciplines  is  concerned  with  nan  and  his  environnent;  each  group 
does,  however,  bring  special  techniques  to  bear  on  the  problen. 

The  industrial  engineer's  tools  are  tine  and  notion  study 
and  nicronotion  analysis.  He  is  also  fanlllar  with  the  technical 
diffculties  facing  design  engineers  In  other  areas  of  specializa t ion. 
In  addition,  he  understands  nanufacturing  techniques  and  production 
problens.  The  industrial  engineer's  special  sVrills  equip  hin  to 
obtain  particular  kinds  of  Infornatlon  about  operator  behavior. 

He  is  in  a  favorable  position  to  help  compronise  conflicts  between 
the  desired  and  obtainable  design. 

The  experiiwental  psychologist  has  an  appreciation  of  the 
intricacies  of  the  human  mechanism,  its  physiological  limitations, 
and  the  factors  which  influence  human  behavior.  He  has  a  thorough 
background  in  scientific  methods,  the  design  of  experiments,  the 
analysis  of  data,  and  the  interpretation  of  results.  J^oreover , 
the  psychologist  has  learned  to  accept  human  ■beha’^dor  for  what  it 
is,  rather  than  worry  about  what  it  should  be.  By  exploring  the 
ways  that  humans  behave  in  controlling  machines  and  working  with 
eaoh  other  as  a  team,  the  psychologist  discovers  the  facts  which 
permit  him  to  modify  conditions  so  that  the  desired  result  is 
obtained • 
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4.0  FR0RL!]1\S  IN  RESFi.RCH  I^.THODOIOGY 


Before  turning  to  a  discussion  of  the  particular  systen  under 
study,  thp  airborne  CIC,  consideration  should  be  given  to  one 
additloral  natter:  r''search  nethods  for  studying  systens  in 
general.  V/hile  each  of  the  issues  to  be  discussed  has  obviously 
been  net  in  previous  research,  there  does  not  seen  to  be  a  source 
which  treats  the  overall  n^- thodolopy  proM  en  as  it  pertains  to 
study  of  a  systen.  This  section  is  intended  to  fill  that  lack. 

The  fundanental  paradign  which  struetures  systen  research  is 
the  faniliar-stinulus ,  organlsn,  and  response.  The  stlnulus  is 
coT"plex.  The  organlsn,  or  systen,  is  nrde  up  of  a  nunber  of 
sub-systems,  each  of  which  nay  be  composed  of  a  wan  and  a  machine. 
The  response,  or  system  output,  is  also  complex;  it  can  he  measured 
in  several  ways,  ’’Vhich  of  these  measurements  can  serve  as  criteria 
of  systen  effectiveness  is  another  question. 

The-e  are  a  number  of  criteria  of  effe^'tive  rpsearch:  that 
it  deal  vith  pertinent  problems;  the  it  be  objective;  and  that 
th,e  results  be  such  that  it  is  possible  to  predict  what  night 
hanpen  under  other  eonditions  which  are  slwilar.  By  pertinent  is 
meant  th;  t  the  crucial  questions  in  any  research  a'^ea  are  found 
and  answered.  An  objective  result  is  one  that  has  not  been  biased 
by  the  nerson  making  the  observation.  Although  this  criterion  v;ill 
not  be  fully  met  in  any  observation,  an  objective  answer  can  be 
defined  as  one  that  can  also  be  found  by  another  experimenter 
working  wi+h  different  subjects  in  a  different  place  and  at  another 
time,  A  result  should  also  have  general  predictive  value;  it  should 
not  be  so  particular  either  to  laboratory  conditions  or  to  specific 


operating  conditions  that  is  not  generally  useful. 

fuch  criteria  are  fairly  easy  to  state  but  can  be  only 
partially  realized  at  best,  and  only  then  by  the  nost  painstaking, 
and  well-considered  work  on  the  part  of  the  scientist. 

These  criteria  nay  be  useful  in  distinguishing  between  applied 
research,  which  is  the  type  called  for  In  this  contract,  and  cure 
reseaich.  In  applied  research,  the  agency  that  buys  the  scientist’s 
services  participates  in  the  deternina t ion  of  ’’pertinency."  It 
would  be  a  nistake,  however,  if  the  contracting  agency  alone 
contributed  to  this  decision.  In  that  case,  the  scientist  might 
be  called  on  to  administer  to  the  symptom  rather  than  the  disease. 
The  full  utilization  of  the  scientist's  capacity  depends  on  making 
use  of  his  perspective.  Discovering  pertinent  questions  for  ex¬ 
perimentation  In  systems  research  demands  mutual  understanding 
and  respect  on  the  part  of  the  scientist  and  the  operating  T^an, 
each  for  the  other. 

There  is  usually  an  urgency  about  applied  research  to  which 
the  scientist  must  be  sensltiveo  This  often  dictates  that  he  try 
to  answer  immediate  rather  than  long-term  nurstlons.  The  scientist 
and  the  operating  man  can  resolve  this  controversy  satisfactorily 
only  in  terms  of  particular  situations,  not  in  terms  of  a  principle. 

Ob.lectivlty  must  sometimes  be  com.promised  as  well.  The 
experienced  applied  scientist  is  willing  to  make  "educated  guesses" 
where  his  observations  provide  hjm  with  co^'pelling  evidence.  This 
is  true  in  proportion  to  the  number  of  th^  scientist’s  colleagues 
who  will  agree  with  him.  It  must  be  remembered  that  without  experi¬ 
mental  results  the  scientist  has  a  batting  average  at  predicting 
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hupon  behavior  only  a  few  points  higher,  at  best,  than  anyone 
else.  A  prediction  is  not  scientific  because  a  nan  whose 
profession  is  science  says  so,  but  because  it  is  based  on  fact. 

It  is  always  desirable  in  applied  work  to  have  results  that 
are  useful  in  predicting  perforrance  under  general  rather  than 
specific  conditions.  Vlienever  a  question  can  be  frared  in  such 
a  way  that  the  answer  will  apply  not  only  to  an  airborne  CIC  but 
also  to  a  shipboard  CIC  and  to  radar  operation  at  large,  nuch  has 
been  gained.  If  the  result  has  predlcti''^e  value  for  another  system, 
yet  unborn,  it  will  be  that  nuch  better.  The  applied  scientist 
cannot  always  deal  with  fundamental  questions  whose  answers  have 
predictive  value  for  many  different  situations;  in  order  to  get 
quick  answers  he  sometimes  has  to  tackle  very  specific  questions. 

He  would  prefer,  however,  to  deal  with  as  fundamental  question  as 
is  consistent  with  the  client’s  need. 

The  human  engineering  study  o'^  systems  of  men  and  machines 
is  largely  an  uncharted  field.  It  can  be  understood,  therefore, 
that  a  human  engineer  is  Interested  not  only  in  answers  to  particular 
questions  but  also  in  discovering  useful  techniques.  This  is  hardly 
an  imuractical  research  product,  especially  for  a  contracting 
agency  that  will  undoubtedly  rerulre  similar  research  on  other 
systems. 

Research  in  a  new  area  usually  proceeds  in  several  stages. 

At  first  the  experimenter  simply  observes  the  phenomenon  he  is 
to  study,  in  this  case  the  performance  of  the  airborne  CIC.  He 
knows  in  general  v.’hat  the  airborne  CIC  is  supposed  to  do,  but  he 
doesn't  know  how  different  operating  procedures  will  affect  system 
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performance.  Nor  d'oes  he  know  how  dlf'^erent  aspect f  of  the  Input 
will  affect  output.  But  as  he  observes,  he  begins  to  have  ideas 
of  his  ov/n  about  v’hat  is  happ^^ning.  At  a  subsequent  stage  of 
investigation,  the  scientist  begins  preliminary  oyperimentatlon 
by  systematically  changing  what  he  considers  to  be  input  to  see 
what  happens.  Are  there  any  changes  in  output?  Is  there  a  linear 
decrement,  if  any,  in  a  certain  output,  or  is  that  decrement 
exponential?  What  is  the  general  functional  relation  between 
aspeets  of  the  output  and  those  of  the  input? 

In  the  same  ’”ay,  he  manipulates  a  system  variable  to  find  the 
relation  between  that  variable  and  output.  In  later  stages  of 
research,  when  the  crucial  variables  have  been  identified  and  the 
functional  relations  among  them  have  been  determined,  the  experi¬ 
menter  may  try  to  get  a  precise  estimate  of  the  parameters  of 
those  functions. 

It  might  be  po:’nted  out  that  few  areas  of  human  knowledge 
are  considered  so  unexplored  that  investigations  are  proceeding  at 
the  preliminary  observation  level.  Research  in  most  areas  involves 
the  study  of  the  relations  between  the  important  variables  which 
have  been  identified.  Human  engineering  study  of  systems  is 
devoted  for  the  most  part  to  the  earlier  phases  of  research: 
the  Identification  of  important  variables — the  structuring  of  the 
domain — and  the  discovery  of  the  funetjonal  relation  between  those 
variables  and  outputc 

As  preliminary  observation  of  a  system's  operation  is  made, 
the  scientist  will  begin  to  list  the  ■'’arious  factors  which  he 
thinks  affect  system  performance.  He  relies  on  his  observations, 
on  a  limited  amount  of  data,  and  on  his  previous  experience  in  study- 
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inp  other  pher.onenr .  Once  the  scientist  ^eels  that  be  has  located 
the  principal  variables  in  syste*^  operation,  he  ran  start  to  classify 
thern  into  logical  grouos.  A  classification  sche^'e  helps  to  structure 
the  problen  and  to  provide  a  reference  ^rane  for  a  research  program. 
The  initial  listing  and  classi^'ica tion  of  variables  must, 
however,  be  considered  as  tentative.  As  study  of  the  system 
continues,  the  classification  scheme  must  be  re-evalua ^ed ,  New 
variables  will  be  added  and  others  dropped.  Research  in  a  new 
area  can  ^e  productive  only  if  the  scientist  maintains  an  open  mind 
about  the  structure  of  that  field,  incorporating  into  the  structure 
the  results  of  experience  gained  as  ■*‘hc  study  progresses. 

The  results  obtained  from  experiments  conducted  in  this  frame- 
’^ork  answer  the  qu-'^stion:  in  what  different  ways  can  the  system  be 
structured  or  why  does  the  corresponding  system  performance  vary? 

The  c'-perlmenter  must  not,  however,  lose  sight  of  the  question? 
which  system  performs  best?  This  means,  of  course,  that  the 
self ntlst  must  be  concerned  with  finding  criteria  of  system  per¬ 
formance  and  designing  a  system  ^hat  will  best  meet  these  crlteriao 
This  is  one  of  the  difficulties  in  systems  research — to  find 
the  best  system  while  also  discovering  vliy  that  system  performs  so 
effectively,  ^ile  these  alms  are  not  completely  lndependent--that 
is,  one  cannot  meet  one  without  partially  meeting  the  other-“the 
two  are  difficult  to  arrive  at  simultaneously.  Both  objectives 
have  very  practical  results:  to  find  which  system  is  perhaps  a 
more  immediate  problem,  while  finding  why  the  system  performs  so 
is  a  long-range  problem.  The  airborne  CIC  is,  of  course,  in  a 
state  of  de'''elopment ,  as  is  the  larger  system  of  wblrh  it  is  a  part. 


It  is  essenti&l  to  :"ind  out  enoufh  about  the  vay  the  systen 
op^^rates  sc  as  to  be  able  to  desjpn  a  sys^ew  which  will  perform 
slightly  different  functions  under  different  conditions.  The 
research  results  should  describe  the  Intricacies  of  the  airborne 
CIC's  operation.  The  engineer  faced  with  designing  a  vastly 
superior  airborne  CIC  should  oe  able  to  discover  fron  these 
results  v;here  the  systen  requires  drastic  improvenent. 

Each  of  the  issues  involved  in  the  questions  raised  in 
previous  pages  must  be  net  in  studying  a  systen  experimentally. 

It  will  be  noted  that  considerable  human  judgment  is  called  for  at 
each  succeeding  stage  of  investigation.  That  is,  inferences  have  been 
drawn  as  to  how  Input  and  output  can  be  measured  and  what  the  system 
variables  are.  ifis  research  progresses,  however,  there  are  many 
points  at  which  the  adequacy  of  those  judgments  can  be  checked. 

If  systen  performance  can  be  p"*edicted  with  reasonable  accuracy 
in  terns  of  these  variables  and  constructs,  these  judgments  are 
adequate. 

4.1  The  Nature  of  the  Stimulus 

A  scientist  is  always  faced  with  two  problems:  discovering 
the  population  of  stimuli  that  impinge  on  the  organism  he  is 
studying,  and  sampling  frow  that  population  in  such  a  way  that 
his  results  will  have  general  predictive  value.  It  can  be 
recognized  that  the  population  of  stimulus  conditions  facing 
an  airborne  CIC  is  hypothetical.  It  is  true  that  there  is  a  real 
population  of  stimulus  conditions  "^hat  have  faced  CICs  in  the 
past,  nth  some  practical  difficulty,  this  population  could 
be  defined  and  samples  taken  from  it.  This  would,  in  effect, 
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test  the  adeouacy  of  the  newly  developed  system  to  handle  a 
past  situation,  whereas  the  research  ain  is  to  d'-sign  a  systen 
that  will  work  for  future  situations.  Only  the  wildest  sort 
of  guess  can  be  nade  about  then.  Axithorlta tive  estinates  as 
to  the  nature  and  strength  of  the  attacks  that  night  be  expected 
by  this  systen  have,  of  course,  been  obtained.  In  general,  the 
load  that  is  forecast  is  considerabh  higiier  than  that  which  has 
been  experienced  by  present  operational  units.  Aspects,  other 
than  load,  distinguish  predicted  from  past  operational  situations. 

There  is  no  need  to  use  a  stimulus  that  represents  such  a 
high  load  that  conrlete  and  utter  confusion  will  prevail  no  matter 
how  the  present  system  is  changed.  Nor  is  there  any  profit  in 
using  a  stimulus  that  represents  such  a  light  load  that  any 
arrangement  of  the  present  syste’^  produces  good  results. 

'.’.Tiat  is  desired  is  a  stl»^u!lus  load  somewhere  between  these 
two  extremes  w’lch  yields  a  "spread”  of  results.  That  Is,  a  load 
condition  under  which  performance  of  the  system  differs  depending 
on  the  system  variable  used.  This  stimulus  load  conaltlon  will 
prove  valuable  unless,  of  course,  it  is  much  lower  than  the  range 
of  predicted  operational  loads. 

A  different  range  of  stimulus  load  might  be  required  If  it 
was  necessary  to  obtain  the  load  limit  that  a  particular  system 
could  handle.  Implicit  in  this  discussion  of  load  conditions  is 
the  fact  that  the  experimenter  must  be  able  to  control  load  condi¬ 
tions.  He  must  either  use  equivalent  loads  to  test  the  effective¬ 
ness  of  different  systems  or  be  able  to  compare  outnuts  under  dif¬ 
ferent  load  conditions  by  balancing  out  the  differences  In  input. 


In  practical  terns,  it  is  pointless  to  de-^'clop  an  airborne 
CIC  ”'hich  can  turn  out  more  infor’^atlon  than  the  control  unit  of 
its  present  system  can  handle.  While  this  larger  system  Is  being 
studied  intensively  In  its  various  aspects,  such  inequalities  are 
tolr-rable.  rome  research  coordinating  agency  must,  however, 
recognize  the  demonstrably  weak  links  in  the  larger  system  and 
take  steps  to  bring  the  develo'-^ental  program  into  line. 

4.2  The  Nature  of  the  Crganism 

To  develop  the  best  system,  or  organism,  the  outputs  of  many 
variations  of  the  same  basic  system  are  compared  under  controlled 
stimulus  conditions.  But  how  can  this  basic  system  be  changed? 
First,  there  are  behavioral  changes  that  can  be  made  in  It;  that 
is,  different  operating  procedures  can  be  used.  Second,  environ¬ 
mental  conditions  can  be  "’odlfled — the  lighting  level,  the  tempera¬ 
ture,  noise  level,  etcc  Third,  there  can  be  design  changes;  the 
equip'^ent  can  be  rearranged,  or  the  content  of  a  display  board  can 
be  reorganized  or  coded  differently. 

The  best  system  is  obviously  that  which  has  the  best  form 
of  each  system  variable;  to  find  the  best  system,  various  forms 
of  each  variable  must  be  compared.  The  question  immediately  arises 
Hov;  can  the  effects  of  these  many  system  variables  be  explored? 

It  would  be  much  easier  from  the  experimental  point  of  view 
to  find  the  best  form  of  one  '''ariable  by  studying  it  in  Isolation 
from  the  others.  But  ’vlll  the  best  form  o^  each  variable — found 
in  separate,  Isolated  experiments- — combine  into  the  best  syrtem? 

It  would,  of  cours'^,  if  its  effects  were  completely  independent. 
That  Is,  the  best  form  of  one  variable  may  '"e  dependent  on  the  form 
of  mother  variable.  This  possible  interdependence  o:^  variable  can 


best  be  illustrated  by  an  example-  Suppose  in  one  study  it  is  found 
that  so  far  as  talking;  procedures  are  concerned  sele^'tive  sv/itching 
technique  is  better  than  a  round  robin  technique.  (Those  trrps  are 
more  fully  explained  in  Section  6.0  where  the  system  variables  are 
listed .) 

Suppose  also  that  the  direct  reporting  procedure  is  better 
than  the  indirect  reporting  procedure.  However,  it  might  be 
that  using  a  direct  reporting  procedure  in  combination  with  a 
selective  switching  talking  pro'^edure  would  be  an  impossible  com¬ 
bination,  that  these  two  procedures  simply  do  not  work  together. 

The  possible  Interdependence  of  variables  mitigates  against 
studying  the  variables  one  by  one.  V/hy  not,  then,  study  all 
reasonable  forms  of  each  system  variable  in  one  experiment?  This 
v/ould  require  a  very  complicated  exnerl’^ental  design,  and  the  con¬ 
duct  of  this  experiment  would  take  a  long  tine.  (It  would  be  a 
case  of  putting  all  one’s  eggs  in  one  basket.)  At  the  very  time 
when  the  scientist's  experience  with  the  system  is  ll-^lted,  he 
’  ould  be  committing  hlmsel^  to  an  extensive  r-xperl’^ent  that  would 
provide  no  answers  at  all  for  perhaps  a  year  or  more. 

It  is  suggested  that  a  co’^promjse  between  those  two  extremes — 
isolated  study  of  '•ach  variable  and  a  simultaneous  study  of  all 
variables — might  be  used  while  exrloring  system  operation.  But 
if  more  than  one  variable  is  to  be  stur’ied  at  a  time,  how  should 
one  select  those  variables  to  be  included  in  any  one  experiment? 

In  terms  of  the  previous  discussion,  it  would  seem  that  the 
scientist  would  study  those  variables  which,  in  his  judgment 
are  Intrrdependent .  This  is,  of  course,  a  very  general  principle 
and  one  that  would  be  hard  to  Implement.  To  a  ccrtalr  extent. 


each  variable  will  depend  on  ea^h  other  variable.  It  should  be 
possible,  however,  for  the  scientists  working  on  the  problem 
to  agree  on  combinations  of  variables  that  seem  to  be  very  closely 
related.  This  .judgment  should  be  based  on  accumulated  experience. 

It  will  be  noted  that  as  study  of  a  system  proceeds  there 
are  fewer  system  variables.  During  preliminary  observation  the 
scientist  selects  out  of  all  possible  variables  in  human  performance 
only  these  which  seem  to  have  to  do  with  system  operation.  During 
exploratory  experimentation,  the  number  of  important  variables  is 
further  reduced.  The  process  should  lead  to  a  crucial  experiment 
where  all  the  Important  variables  that  remain  are  studied  simul¬ 
taneously.  This  crucial  experiment  will  be  more  complicated  than 
the  exploratory  ones  that  preceded  it.  The  scientist  will,  at  that 
point,  however,  ha^e  a  good  deal  of  experience  with  system 
operation,  will  have  refined  his  classification  of  system  variables, 
and  will  have  formulated  a  series  of  critical  Questions  which 
warrant  the  extensive  experiment. 

There  remains  the  problem  of  fixing  the  constant  values  of 
those  variables  not  being  manipulated  in  any  s+udy.  Although  the 
variables  held  constant  are  believed  to  be  independent  of  those 
studied,  values  for  them  should  be  found  that  facilitate  stable 
operation  of  the  system.  It  is  suggested  that  the  experimenter 
specify  for  the  crew  only  general  operating  alternatives.  Through 
the  exercise  of  initiative  and  resourcefulness,  the  crew  in 
preliminary  runs  will  find  appropriate  values  for  those  constant 
variables. 

The  crew  Itself  is  an  important  variable  in  system  performance 
in  addition  to  the  behavioral,  environmental  and  design  variables. 
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Again  there  is  the  problem  of  defining  a  population,  this  time 
of  crew  members.  There  is  a  real  population  of  CIC  crew  members 
at  present,  but  the  copulation  v/hieh  v'ould  exist  under  emergency 
conditions  is  hypo\ httical.  Fany  men  who  are  now  ci-^^ilians  would 
become  CIC  crew  members;  who  they  are  and  what  they  are  like  is 
not  known.  Adequacy  of  sampling  of  operators  is  not  as  crucial 
to  human  engineering  work  as  it  would  be  to  research  on  selection 
or  training  of  those  men.  As  a  matter  of  fac*  what  is  desired 
of  a  crew  working  under  experimental  conditions  might  not  be  found 
in  a  representative  sample.  The  crew  member  should  have  CIC  back¬ 
ground,  some  scope  experience,  an  interest  in  the  work,  and  should 
be  able  to  avoid  strong  personal  preferences  for  particular  ways 
of  operating.  The  subject  should  be  able  to  perform  to  his  utmost 
under  any  condition  that  is  specified. 

,  The  especially  proficient  operator  is  not  wanted  because  he 
has  already  developed  a  "style”  which  he  has  completely  mastered. 
The  crew  member  on  this  project  is  not  being  trained  for  a  CIC 
Job,  but  is  serving  as  an  experimental  subject.  He  may  even  turn 
out  to  be  less  capable  than  a  newly  trained  crew  member.  The  new 
crew  member  will  have  been  trained  to  a  particular  set  of  operating 
procedures  while  the  experimental  subject  will  have  been  operating 
in  many  different  v-ays.  The  man  v;ho  has  served  as  an  experimental 
subject  will  have  to  unlearn  many  behaviors  that  are  no  longer 
wanted  under  operational  conditions. 

It  is  important  to  desjgn  system  studies  in  such  a  way  that 
individual  differences  in  the  present  crev^  are  balanced  out.  This 
Involves  rotation  of  the  individuals  through  the  various  crew 
positions  in  some  systematic  v/ay.  At  present,  it  is  possible 


to  rotate  the  Kaval  officers  through  those  positions  v;hi^h  are  to 
be  filled  by  officers.  The  enMsted  nan  conplonent  is  not  large 
enough  to  permit  this.  A  request  has  been  made  for  additional 
enlisted  nrn  so  that  this  rotation  of  individuals  through  Jobs 
will  be  possible. 

The  fact  that  the  exnerinental  crev'  is  not  a  representative 
sample  of  the  population  of  future  CIC  crew  members  prohibits 
prediction  of  the  exact  parameters  of  functional  relations. 

This  does  not  prevent  the  study  o:^  the  nature  of  those  functions, 
however.  There  are  a  number  of  other  factors  Inherent  in  the 
laboratory  situation  which  prevent  the  determination  of  exact 
parameters;  Inadequacy  of  sampling  of  the  crew  nay  be  one  of  the 
lesser  evils  in  this  respect, 

4, 3  The  Nature  of  Measurement 

The  first  step  in  measuring  output  is  to  consult  with 
operational  organizations  to  find  what  is  expected  from  the  system. 
Preliminary  observations  may  then  be  made  of  system  output  to 
see  what  sort  of  measurements  can  be  made. 

Applied  research  differs  from  pure  research  in  part  in  the 
matter  of  the  unit  of  measurement.  The  molar  unit  of  measurement 
is  required  here.  A  molar  unit  is  coi^prlsed  of  a  number  of 
smaller  units.  For  example,  there  are  these  time  elements  in 
reporting  a  new  raid: 
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Element  Description 

A  From  the  tire  a  tarret  pets  rithin  range  until  the 
radar  sv'eep  gets  to  the  azimuth 

B  From  A  until  the  operator  notices  it 

C  From  B  until  he’s  sure  its  a  target  and  rot  noise 

D  Fror  C  until  he  reads  its  posj't'ion 

E  From  D  until  he  can  throw  the  sv'itch  on  his  ICS 

F  Fro*”  E  until  he  gets  his  party 

G  From  F  until  he  finishes  his  report 

H  From  G  until  the  Status  Board  Keeper  records  it 

I  From  H  until  the  Talker  can  notice  it 

J  From  I  until  the  Talki^r  contacts  OTC 

K  From  J  until  the  Talker  starts  the  report 

These  small  elements  can  he  combined  into  the  molar  unit: 
time  for  a  system  to  detect  a  new  target.  A  molar  unit  is  one  of 
practioal  usefulness  appropriate  to  the  quality  being  measured.  A 
man’s  height,  for  instance,  is  measured  in  feet  and  inches  rather 
than  in  thousands  of  an  inch.  The  ultimate  test  of  the  appropriate¬ 
ness  of  a  molar  unit  is  whether  predi^tinns  can  be  made  using  it. 

There  is  another  determinant  of  th*"  size  of  a  unit,  and  that 
is  the  extent  of  instrumentation  neces'^ary  to  obtain  it.  To  measure 
some  elements  in  the  example  cited  would  be  a  difficult  task. 

It  is  necessary  In  applied  v^ork  ■t'o  deal  with  data  which  is  relatively 
easy  to  collect  and  whach  is  amenable  to  rapid  analysis. 

Thus  far,  the  measurement  of  syr^tem  output  alone  has  been 
considered.  There  is  an  additional  way  In  which  the  experimenter 
insures  the  value  of  his  study.  That  is  by  describing  as  many  of 
the  internal  workings  of  the  system  as  he  can.  If  variation  In 
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perf orronce  of  the  system  cannot  be  accounted  for  in  terms  of 
the  design  variables,  the  experimenter  must  try  to  find  other 
reasons.  There  are  a  pood  many  aspects  of  th-"'  behavior  of  the 
crev7  members  which  cannot  be  specified.  iVhile  the  CIC  officer  might 
be  instructed  to  direct  most  of  his  attention  to  a  particular 
display,  it  is  improbable  that  under  the  stress  of  a  problem  he 
could  look  at  Display  A  four  times  as  often  as  Display  B  (in  carry¬ 
ing  out  his  instructions),  without  seriously  hampering  his  perfor^anc 
Activity  analysis  of  individual  behavior  can  be  made;  records  of 
ervironmcntel  (Conditions  can  be  kept.  The  experimenter  can  then 
correlate  performance  variations  with  these  records  to  see  if  there 
is  a  strong  relation  between  output  and  the  indicators  of  Internal 
activity.  This  procedure  provides  a  continuing  check  on  the  ap¬ 
propriateness  of  the  variables  that  are  being  used  in  the  ex¬ 
perimental  design.  It  will  also  provide  insights  of  the  basis 
on  which  additional  experiments  can  be  designed. 

In  problems  of  measurement  it  is  often  necessary  to  make 
certain  assumptions  regarding  the  nature  of  errors  in  order  that 
a  rationale  for  measurement  can  be  developed.  This  will  also  be 
the  case  in  ‘^vstem  measurement.  Such  assumptions  involve  the 
randomness  of  certain  errors  as  contrasted  to  constant  or  systematic 
errors  which  must  be  treated  separately. 

It  is  necessary  to  order  the  resulting  data  of  systems  study 
in  such  a  way  that  they  can  be  combined  with  data  available  from 
other  sources,  A  laboratory  study  does  not  provide  information 
on  the  performance  of  the  radar  gear  itself.  Nor  does  it  provide 
data  on  the  detection  of  pips  by  the  operator  v^orking  off  a  true 
radar  picture  where  noise  is  distracting,  ^lle  the  target 


generators  whleh  represent  a  GAP  do  fail,  they  do  not  do  so  with 
the  same  probability  that  an  aircraft  does.  Ultimately,  the 
data  on  the  system  must  be  combined  with  that  of  the  other  links. 
This  can  be  done  only  in  later  stages  of  the  study,  but  the  ex¬ 
perimenter  must  have  his  results  in  the  form  that  lends  itself  to 
that  process. 

4.4  Design  of  System  Experiments 

The  matter  of  incorporating  system  variables  and  crew  member 
differences  into  experimental  design  has  already  been  discussed. 
There  are  several  other  factors  which  also  must  be  considered  in 
designing  an  experiment.  Do  certain  operating  nrocedures  work 
well  under  low  load  but  not  under  high  load?  The  system,  variables 
should  be  tested  over  a  range  of  stimulus  loads.  Is  there  a 
marked  decrement  in  output  with  some  operating  procedures  as  the 
system  continues  to  perform?  The  system  should  best  be  tested  over 
a  period  of  time  to  see  whether  performance  falls  off  more  rapidly 
with  some  operating  procedures  than  with  others. 

There  are  two  factors  almost  too  obvious  to  mention.  Human 
engineering  study  is  not  so  much  concerned  with  finding  how 
quickly  operating  conditions  can  be  learned  as  with  comparing  them 
once  they  have  been  mastered.  Therefore,  sufficient  ] ractice  in 
any  operating  condition  is  given  to  bring  the  r  ew  past  the  initial 
learning  stage.  However,  the  experimenter  should  check  to  see 
whether  performance  is  stable  or  whether  it  continues  to  improve 
throughout  the  experiment.  He  should  also  balance  out  the  order 
in  v'hleh  various  operating  conditions  are  used  so  that  differences 
in  performance  can  be  attributed  to  the  procedures  themselves 
rather  -^han  to  the  order  of  presentation. 


The  object  of  the  exploratory  experiments  is  to  rather  data 
rapic’ly  on  the  effe-^ts  of  all  the  system  variables.  Tiat  sort  of 
Gxrerimental  r’esign  lends  itself  to  this  purpose?  Tliere  are,  of 
course,  many  experimental  designs,  each  of  ^-bich  has  its  particular 
advantages.  A  factorial  design  yields  the  most  complete  information, 
including  detailed  estimates  of  the  interactions  between  the  vari¬ 
ables  in  the  design;  it  is  also  the  most  time  consuming.  The 
Graeco-Latin  square  design  is  less  time  consuming,  but  does  not 
provide  as  much  information  on  the  interactions  between  variables. 

An  incomplete  block  design  is  very  s.  ..ir  to  the  Graeco-Latin 
square  design.  In  it  the  effects  of  some  of  the  variables  are 
systematically  balanced  out.  It  seems  appropriate  during  ex¬ 
ploratory  experimentation  to  utilize  either  of  the  latter  designs, 
sacrificing  detailed  knowledge  of  the  interaction  effects  to  the 
rapid  accumulation  of  results. 

In  testing  the  differen'-es  in  performance  under  various 
operating  conditions,  the  statistical  results  could  be  wrong  in 
either  of  two  ’’'ays.  There  could-be  a  real  difference  in  the 
performance  between  two  conditions  which  would  not  be  shown  as 
statistically  significant.  Or  there  could  be  no  real  difference 
in  the  two  conditions,  one  of  wliioh  v'ould  be  showh  to  be  signifi¬ 
cantly  better  than  the  other.  In  system  study,  an  effort  is  being 
made  to  find  new  and  better  ways  of  operating.  In  terms  of  the 
purpose  of  the  research,  it  v’ould  be  better  to  infer  that  one 
operating  procedure  was  better  t}>an  another  even  if  it  were  not, 
rather  than  to  discard  a  procedure  because  it  did  not  appear  to 
give,  statistically,  a  significantly  better  result  when  actually 
it  was  superior. 
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This  research  attitude  has  inpllcations  for  experinental 
design.  The  desire  to  err  in  the  direction  of  finding  dif¬ 
ferences  where  there  are  none,  rather  than  vice  ve'  sa ,  indicates 
that  as  precise  a  measure  of  experimental  error  as  possible  must 
be  obtained.  This  means  that  data  on  several  runs  under  the 
same  condition  should  be  obtained  and  that  the  error  estimate 
should  be  reduced  by  those  factors  which  seem  to  contribute  to 
a  lack  of  homogeneity. 

The  need  for  a  final  crucial  experiment  has  been  indicated. 
This  experiment  might  take  one  of  several  forms.  A  very  rough 
check  on  the  previous  exploratory  experiments  might  be  made  by 
comparing  only  two  systems:  one  which  represented  the  better 
condition  for  each  variable,  and  the  other  which  represented  the 
poorer  condition.  This  would  provide  a  quick  test  of  the  validity 
of  the  general  results.  It  would  hardly  be  conclusive,  however. 

If  the  difference  between  the  outputs  of  the  two  systems  were 
small,  it  would  be  almost  impossible  to  isolate  the  causes. 

Another  possibility,  to  which  the  previous  discussions  have 
been  pointed,  is  to  conduct  a  final  experiment  of  a  factorial 
design  on  the  refined  list  of  variables  obtained  from  the 
exploratory  experiments.  A  tenta'^ive  comparison  of  the  exploratory 
results  would  be  helpful  for  the  design  of  a  crucial  experiment. 
This  could  be  done  by  maintaining  a  common  element  throughout 
the  series  of  exploratory  experiments.  This  common  element  would 
be  performance  of  the  system  under  a  standard  condition,  which  would 
have  to  include  both  a  standard  operating  condition  and  a  standard 
stimulus  condition.  The  tabulation  of  results  of  the  series  of 
experiments  would  take  this  form: 
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This  exaniple  is  a  consicerably  sinplified  vppsjon  of  the  conplicated 
table  that  vould  result.  It  vill  be  noted  that  the  standard  condition 
is  the  only  one  cownon  to  all  experinents,  and  that  there  are  inany 
blanks  in  the  table.  But,  by  taking  into  account  the  difference 
between  X2g  and  the  exnerir’enter  can  get  a  tentative  comparison 

of  perfor’^ance  under  Conditions  B  and  C.  V'hile  this  table  mgy  not 
lend  itself  to  a  statistical  analysis,  there  are  many  inferences 
that  can  be  dravn  from  it.  It  pro'^^ides  the  experimenter  with  a 
much  better  basis 'for  integrating  the  results  of  the  exploratory 
experiments  and  designing  the  crucial  one.  Running  the  system  under 
a  standard  condition  in  each  experiment  is,  of  course,  the  key  to 
this  sort  of  comparison. 

There  is  one  final  can, cion  that  should  he  given.  Diiring  the 
early  Thirties  an  experiment  was  run  at  the  Hawthorne  plant  of 
the  t'Gstern  F-lectric  Company.  There  the  production  of  a  small 
group  was  compared  under  different  working  conditions.  It  was 
found  that  the  fact  that  this  group  was  gi  special  attention 
caused  production  to  increase  irrespective  of  what  working 
conditions  v’ere  used.  This  has  sometimes  been  called  "the 
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Hawthorne  effect o”  The  scientist  nust  he  f'oncerned  that  the 


improvenent  of  the  performance  of  any  system  is  due  to  better 
operating  conditions  rather  than  to  the  inp:"Oved  morale  that 
result  a  from  the  special  attention  given  to  an  experimental . group. 
It  may  be  that  the  crew  will  become  so  highly  motivated  and  so 
skilled  that  they  can  perform  almost  equally  well  under  the 
various  operating  conditions  that  are  tested.  This  is  another 
reason  why  the  experimenter  must  obtain  precise  measures  of 
experimental,  error  so  that  he  can  find  any  differences  in  the 
effectiveness  of  operation  procedures  that  may  exist. 

4.  5  Summary  of  Methodological  Principles 

The  previous  discussion  of  research  methodology  can  be  sum¬ 
marized  in  the  methodological  principles  which  follow: 

4.5.1  General: 

a)  The  basic  model  for  system  study  is:  stimulus,  organism 
response. 

b)  The  criteria  of  effective  research  are  pertinency, 
objectivity  and  the  "generalize-ability"  of  the  results. 

c)  In  applied  research: 

1.  Pertinency  is  determined  in  part  by  the  purchaser 
of  the  research. 

2.  The  seientist  sometimes  has  to  make  "eou'^ated 
guesses”  vhi^h  are  a  compromise  of  objectivity. 

I^cT’e  often  specific  rather  than  general  ouestlons 
must  be  answered. 

Urgency  so^'etimos  renulres  immediate  rather  than 
long-term  answers. 

d)  because  human  engineering  s'f'udy  of  systems  of  men  and 
machines  is  a  new  branch  of  applied  research,  the  scientist  is 
Interested  in  techniques  as  well  as  results. 
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e)  Syston  study  proceeds  In  se^erj-l  s'f'ages:  the 

observe  ticns ,  exploratory  pxperlr’ontatl on ,  and  a  crucial  eypt  rinent. 
Tn  peneral,  the  "functional  relations  hetreen  input  and  output,  and 
those  between  system  variables  and  output  (o'^’er  a  range  of  inputs) 
are  found.  It  may  not  be  possible  to  discover  the  exact  parameters 
of  these  functions. 

f)  The  exp‘ rimenter  must  recognlre  that  the  aim  of  system 
study  is  to  find  the  best  system;  he  must  at  the  same  time  try  to 
find  out  why  that  system  performs  as  it  does  so  that  he  may  project 
design  for  future  situations. 

4.5.2  Regarding  the  Stimulus: 

a)  T*-  is  not  possible  to  sample  from  the  hypothetical  popu¬ 
lation  c:’  stimulus  conditions  that  will  occur  in  the  future.  It 

is  sufficient,  however,  to  select  a  load  level  which  is  a  reasonable 
representation  of  the  stimulus  co’^plex,  so  that  it  produces  a  ranpe 
of  outputs  which  will  distinguish  between  the  effectiveness  of 
the  conditions  tested. 

b)  In  order  that  the  ef fe'^t iveness  of  se^oral  -^'ersions  of 
the  system  can  be  co’-pared,  +he  stimulus  condition  must  be  controlled 
either  by  presenting  enuivalent  stimuli  or  by  balancing  out  the 
di"fferences  in  the  stimuli  by  means  of  the  experimental  design. 

4.5.3  Regarding  the  Organlsirn 

a)  The  system  itself  can  be  modified  by  changing  behavicral, 
environmental,  or  design  conditions. 

b)  As  research  progresses,  the  list  of  system  variables 
must  be  constantly  re-examined.  The  adequacy  of  this  hypothesized 
list  of  variables  can  be  Judged  in  terms  of  how  ivell  system  per¬ 
formance  can  be  predicted  from  it. 


c)  The  interdependence  of  varlaMes  nay  confound  the 
results.  At  least  tv;o  precautions  should  he  observed:  take  care 
to  study  groups  of  honogeneous  va-^lahles,  and  let  the  system  reach 
a  stable  operation  condition  by  permitting  the  crew  members  to 
work  out  the  details  of  general  operating  procedures  in  preliminary 
runs. 

d)  The  population  of  future  CIC  crew  members  is  hypothetical. 
Actually,  a  representative  sample  the  true  population  may  not  be 
desirable  for  experimental  work;  rs'^her,  the  crew  members  should 
meet  these  specifications:  they  should  have  CIC  background  and  some 
scope  experience,  should  be  sufficiently  motivated  and  able  to 
avoid  strong  personal  preferences  for  particular  ways  of  operating. 

Regarding  Measurement: 

a)  The  mjlar  unit  is  the  measure  needed  in  applied  research. 
The  molar  unit  is  larger  than  that  used  in  pure  research;  it  usually 
has  a  practical  meaning;  it  is  appropriate  to  the  quality  being 
measured;  it  can  readily  be  v^btained  and  analyzed. 

b)  It  is  necessary  to  measure  internal  activity  and  envir¬ 
onmental  conditions  as  a  check  on  experimental  adequacy.  By  cor¬ 
relation  techniques,  the  experimenter  can  explore  inductively  the 
causes  for  variance  in  performance  not  otherwise  accounted  for. 

c)  The  experimenter  needs  to  distinguish  between  constant 
and  random  error  on  the  part  of  the  crew  member;  he  has  to  make 
certain  assumptions  about  random  error  in  order  to  get  a 
measurem.ent  rationale. 

d)  The  data  on  system  performance  must  be  so  ordered  that 
they  can  be  co’^blned  with  data  on  other  lines  for  overall 
predictive  purposes. 


4,5»5  Design  Svsten  "^xprr Inents: 

a)  Two  stapes  of  oxn--  ri'^entatlon  are  inculcated:  1)  a 
series  of  s’  ort  exploratory  stuc;ies  of  all  system  variables  In 
honopeneous  groups;  2)  a  crueinl  exp'^rinen't'  In  v’M<^h  the  critical 
variables  are  stuc’led  sinultaneously. 

b)  The  Graoco-Lftin  square  and  jnconple't'e  block  exp^ri- 
nental  designs  soei^  appropriate  for  the  first  stage  of  experi¬ 
mentation  where  the  need  to  aceumulate  results  rapidly  justi-^ies 
■•■he  sacrifice  of  more  detailed  information.  The  more  extensive 
factorial  design  is  appropriate  "^or  the  second  stage. 

cO  A  system  experiment  should  take  into  account  in  its 
design  those  factors:  a  group  of  ho’^ogeneous  system,  variables, 
the  individual  differences  in  the  cre^’,  a  range  o''  stimulus  loads, 
the  fatigue  effeot  of  continued  performance,  learning  effects,  and 
the  order  of  presentation  o"^  op^^ratlnnal  conditions. 

d)  The  nature  of  s;'^stem  research  is  such  that  the  experi¬ 
menter  should  design  his  studies  to  be  sure  to  find  any  differences 
in  performance  ^hat  ^  exist  rather  than  to  avoid  :^lnding  difference 
that  do  not  exist, 

e)  The  use  o:^  a  standard  condition  throughout  exploratory 
experinents  will  permit  tentative  Integration  of  performance  results 
it  will  also  provide  a  bettor  basis  for  the  design  of  a  crucial 
experiment. 

f)  The  rxperimrnter  should  be  cautioned  about  the 

‘'Hawthorne  effect,"  the  tendency  of  an  experimental  group  to 
respond  so  strongly  to  special  attention  that  actual  differences  in 
the  ef fer' i-'-ent ..  "  various  operations  conditions  are  minimized 

or  lost  altogether. 


-  45  - 


5.0  THE  STI?.tULUS  COT^^PLEX 


It  Is  necessary  to  classify  the  elements  of  the  system 
stim.ulus  as  a  first  step  in  determining  pxpo^  iment  conditions. 

The  detailed  description  of  the  anatomy  of  the  organism,  or  system, 
and  of  the  response,  and  how  it  can  he  measured,  will  follow  in 
succeeding  sections  of  this  report. 

The  stimulus  complex  considered  here  is  that  of  the  exterior 
stimulus,  or  the  input  to  the  system.  There  are,  of  course, 
internal  stimuli  that  impinge  on  the  system,  such  as  the  personali¬ 
ties  of  the  operators,  and  the  continuing  process  by  which  the 
response  of  a  sub-system  serves  as  a  stimulus  to  other  sub-systems 
and  to  the  first  sub-system  itself.  At  this  stage  of  research 
distinguishing  the  respective  feedback  phenomena  can  be  neglected 
for  the  moment  in  favor  of  the  response  of  the  system  as  a  whole  to 
exterior  stimuli. 

5.1  The  General  Nature  of  the  Stimulus 

The  general  relation  of  the  stimulus  coT^plex  to  system 
response  can  be  stated  as  follows: 

^11  “  aiixil  ai2X2i  . nv^rl  (D 

■'■’here  is  the  response,  or  performance  score,  of  System  1  to 

Stimulus  1  ^’hich  is  composed  of  "r"  independent  elements.  Each 

tern  on  the  right  side  of  the  equal  sign  is  made  up  of  two  parts: 

the  "a”  which  represents  an  independent  capacity  of  System  1 

to  deal  v/lth  the  corresponding  element  "x"  of  Stimulus  1.  This 

equation  states  that  there  are  "r”  separate  and  dis'^'inct  qualities 

of  the  stimulus,  to  each  of  -which  the  system  must  respond;  it  also 

says  that  a  system  has  "r"  corresponding  capacities  to  deal  with  a 

stimulus,  each  of  which  may  vary.  The  response  of  System  2 
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(the  sane  systen  usinp'  different  operating  eonditions)  to  the 
sane  stimulus  can  be  described  as  follows: 

^21  "  ®21^11  ®22^21  ®23^31  .  ®2r^rl 

^act  of  the  "x' s"  is  the  sane  as  in  equation  (1)  because  the 
sane  stimulus  is  used,  flach  of  the  "a's"  is  different,  hov’e'''er, 
beeause  there  is  a  different  systen  to  deal  with  that  stimulus. 
The  Independent  capacities  of  that  second  systen  nay  be  different 
from  those  of  the  first. 

The  actual  relation  between  the  stimulus  complex  and  the 
systen  response  may  be  a  good  deal  more  complicated  than  the 
linear  equation  in  r-space.  Fortunately,  hov^ever,  this  linear 
equation  v-ill  apnroxinate  the  true  function.  An  approximation 
such  as  this  will  be  useful  at  the  stage  of  investigation  when 
actual  elements  of  the  stimulus  and  of  the  system's  capacity  are 
not  known. 

A  set  of  equations  can  be  written  for  the  j esponse  of  any 
system  to  any  stimulus.  If  enough  data  are  collected,  this  set 
of  simultaneous  eeuations  can  be  sol''’ed  to  find  the  number  of 
independent  elements  of  the  stimulus,  and  v;hat  they  are.  It  will 
be  more  convenient  if  these  simultaneous  equations  are  stated  in 
matrix  form. 
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This  is  a  shorthand  notation  for  the  series  of  simultaneous 
equations  which  indicates  the  relation  between  the  response 
of  n  systems  to  N  stii^ulus  complexes.  The  general  statement 


of  the  relation  ist 

*  «12*2d  *■  »13*3i  . ®lr*r3 

The  relation  can  be  stated  more  simply  ast 

PF  =  S  (5") 


The  right  side  of  the  equation  contains  the  observable  results, 

or  the  data  that  are  gathered.  How  can  the  equation  be  solved 

once  the  performance  scores,  the  S's,  are  known?  Th^se  answers 

can  be  found  by  means  of  factor  analysis.  The  correlation  matrix, 

« 

R]_,  which  expresses  the  correlation  coefficients  between  pairs 
of  systems  tested  throughout  the  range  of  stimuli  can  be  ob¬ 
tained  in  this  way* 

1  S  S'  =  R,  (6) 

N  ^ 

It  can  be  shown  that  from  the  reduced  correlation  matrix,  R,  the 
population  matrix,  P,  can  be  found  by  factoring  and  rotating  to 
simple  structure.  The  factor  equation  to  be  solved  is; 

F  F '  =  R  (7) 

But  why  should  there  be  such  concern  over  the  theoretical 
relation  between  system  response  and  the  stimulus  complex?  The 
purpose  of  delineating  the  elements  of  the  stimulus  is,  of  course, 
two-fold;  1)  to  comprehend  the  actual  stimulus  thoroughly 
enough  to  be  able  to  present  an  operationally  realistic  problem; 

2)  to  control  the  stimulus  so  that  a  valid  comparison  can  be  made 
among  the  different  operating  conditions  of  the  system.  For, 
unless  the  stimuli  presented  to  different  forms  of  the  system  are 


equivalent,  or  ean  he  s^^aled  against  some  standard,  the  better 
■Terf ormance  of  one  eanrot  be  established  as  superior. 

the  columns  of  the  P  matrix  (see  formula  (3))  were  summed, 
these  values  would  be  obtained. 


. 


(8) 


By  dividing  by  n,  the  eapaoity  of  the  average  system  in  each  of 
the  r  abilities  could  be  found: 


a-i  ap  S';  sa 

or 

®sl,  ®s2,  ®s3,  ®s4. 


a 


sr 


(9) 


(10) 


By  using  these  mean  values  for  each  of  the  abilities,  the  standard 
response  of  the  average  system  eould  be  comnuted  for  any  stimulus: 


®s2^2,1  ®s3^3.1 


«sr^r1 


(11) 


In  this  way,  a  number  of  stimuli  could  be  designed,  the  standard 
responses  to  which  would  range  over  a  continuum  from  low  to  highc 

I  I  I  »  I  I  I 

L  ow _ H  i  gh 

®sl  ®s2  Es4  Sgj  Ss6 

Any  system  could  then  be  tested  against  these  stimulus  conditions. 
The  estimate  of  the  average  system’s  abilities  will  become  more 
precise,  of  course,  as  the  number  of  systems  tested  increases. 

But  v/hat  could  these  elements  of  the  stimulus  condition  be? 

One  element  may  be  space  density,*  or  the  number  of  continuous 


"*Another  construct  similar  to  space  density  which  has  been  used  is 
"target  minutes."  Instead  of  equating  the  number  of  targets  on  the 
scope  for  comparable  periods,  the  number  of  target  minutes  can  be 
equated.  As  its  name  Implies,  a  target  minute  is  the  presence  of  a 
target  on  the  scope  for  one  minute.  By  multiplying  the  number  of 
minutes  each  target  is  on  the  si^ope  and  adding  these  figures  for  all 
targets,  the  target  minute  load  for  a  stimulus  condition  can  be  found 


events  occurring  simultaneously.  Another  night  be  tine  density, 
or  the  nunber  of  discrete  events  occurring  per  unit  time. 

As  a  matter  of  fact,  experimental  results  have  indicated 
+-hat  there  is  a  functional  relation  between  time  density  and  output 
as  well  as  between  space  density  and  output.  These  functional 
relations  prevailed  when  the  perf'^rnance  of  individual  operators 
was  studied.  However,  in  systems  operation  there  is  division  of 
labor  which  apportions  the  total  load  tv/o,  three,  or  four  ways 
insofar  as  scope  watching  is  concernedo  The  functional  relations 
between  these  two  aspects  of  the  stimulus  and  system  output  might 
not  he  consistent  under  present  cpe  rating  conditions  because  of 
the  saturation  of  other  links  in  the  system.  For  example,  the 
physical  capacity  of  the  verbal  display  may  be  much  lower  than 
that  of  other  links  in  the  system.  This  limitation  may  garble  the 
data  at  higher  loads  so  that  only  after  redesign  of  particular 
components  will  it  be  possible  to  get  consistent  data  throughout 
the  range  of  loads. 

A  problem  in  dealing  with  time  density  needs  to  be  mentioned. 
Different  discrete  events  take  varying  amounts  of  time  for  the 
operator  to  detect.  That  is,  a  speed  change  in  a  target  cannot 
be  read  directly  off  the  radar  picture,  but  must  be  Inferred  from 
accumulated  scope  history.  This  means  that  the  operator  can^^ot 
report  the  occurrence  of  a  speed  change  as  rapidly  as  the  ap¬ 
pearance  of  a  new  target.  The  question  then  arises:  should  a  speed 
change  simulated  at  I'^lnute  10  be  considered  a  time  density  element 
at  rinute  10  or  at  I/inute  12 — the  time  at  w>>ich  the  operator  can 
first  detect  such  an  event? 
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The  ma+'ter  of  dtilvlng  a  standard  response  measure  will 
require  the  accumulation  of  much  data.  Whrt  technique  should  be 


used  in  preliminary  vork  to  obtain  a  range  of  stimulus  loads?  It  is 
suggested  that  proportional  parts  of  each  hypothesized  clement  of 
the  stimulus  be  used.  For  Instance,  if  it  is  desired  to  use  four 
different  load  conditions,  1,  2,  3?  and  4,  Load  1  to  be  lighter  than 
2,  2  lighter  than  3?  and  so  on,  the  stimulus  problem  could  be  set 
up  in  this  fashion: 


Load 

Condition 

1 

2 

3 

4 


2. 


5 

10 

15 

20 


9 

12 


4 

8 

12 

16 


2 

4 

6 

8 


It  would  seem  that  Load  4  is  greater  than  Load  3  and  Load  3 
is  greater  than  Load  2.  Although  it  is  a  question  whether  Load  4  is 
as  much  greater  than  Load  3  as  Load  3  is  greater  than  Load  2,  this 
will  provide  a  first  approximation.  Of  course,  the  design  of  these 
stimulus  conditions  is  limited  by  homing  to  use  proportional  parts 
of  each  element. 

It  seems  that  there  are  two  distinguishable  aspects  of  the 
exterior  stimulus  to  the  organism,  or  system:  the  scope  stimulus, 
and  the  verbal  stimulus.  These  two  kinds  of  stimuli  will  be  dis¬ 
cussed  separately  and  in  more  detail  in  the  sections  to  follow, 

5.2  The  Scone  Stimuli 

The  scope  stimuli  are  those  which  are  presented  to  the 
operators  watching  the  radar  displays.  They  can  be  categorized 
into  two  groups:  continuous  and  discrete  stimuli. 

5.2.1  Continuous  Scone  Stimuli:  Included  in  this  category  are 


the  following: 
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Position 

Course 

Speed 

Altitude 

Composition  of  raid 
General  taetioal  situation 

The  first  five  of  these  are  reportable  aspects  of  one 
target  on  a  radar  scope;  the  content  of  the  report  depends  on  the 
moment  when  it  is  made.  The  composition  of  a  raid  is  judged  in 
term.s  of  "plpology"  and  the  initial  range  of  detection,  course, 
speed,  and  altitude  of  the  target.  In  the  laboratory,  "pipology"  is 
not  m(  aningful  because  the  simulated  ’'pips"  are  all  the  same;  the 
remaining  aspects  of  a  raid  can  be  used,  however,  to  judge 
r'ompositlon.  A  composition  report  tells  whether  the  target  is 
surface  or  air,  and  in  the  la‘''ter  case,  how  ’^any  planes  of  what 
type* 

The  general  tactical  situation  is  a  continuously  developing 
condition  determined  by  the  prevailing  relations  among  the  targets 
on  the  scope. 

The  space  density,  or  continuous  stimulus,  load  thus 
becomes  a  function  of  the  number  of  targets  on  the  scope. 

5.2.2  Discrete  Scope  Stimuli;  Included  in  this  category  are 
the  following: 

New  targets 
IFF  signals 

Target  fades  and  reappearances 
Course  changes 
Speed  changes 
Altitude  changes 

Loss  of  target  through  entering  weather 
area,  lea-'^ing  the  area  of  surveillance,  or 
entering  the  sea  return. 

Each  of  these  events  occurs  at  a  specific  point  in  time; 
each  is  distinguishable  from  the  continuous  stimuli  in  that  these 
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repcrts  can  be  co’T^pared  vith  the  time  at  vrhich  the  event  actually 
occurred  and  are  not  a  function  of  either  the  time  at  which  the 
operator  reports  or  the  number  of  targets  on  the  scope.  The 
position  of  a  target,  for  instance,  v/ill  depend  on  whether  the 
operator  Is  reporting  at  Minute  10  or  12,  whereas  the  onset  of  a 
new  target  can  be  reported  as  of  a  particular  time  even  if  the 
operator  cannot  get  his  report  on  the  channel  for  several  minutes. 
5.2.3  Particular  Internal  Scone  Stimuli?  Although,  in  general, 
consideration  of  the  stimuli  is  limjted  to  those  external  inputs 
to  the  system,  there  are  several  scope  stimuli,  internal  to  the 
systei^,  which  are  eritical  to  system  operation.  An  example  of 
this  is  when  a  target  crosses  a  sector  boundary  and  surveillance 
of  it  must  be  transferred  from  one  operator  to  another.  Problem 
inputs  are  so  designed  to  include  a  number  of  these  occurrences 
and  to  keep  the  number  of  these  events  constant  from  one  problem 
to  another.  These  are  continuous  rather  than  discrete  stimuli 
because  the  process  involves  both  anticipation  and  follow-through 
rather  than  an  instantaneous  transfer  when  the  target  crosses  a 
boundary. 

5»  3  The  Verbal  Stimuli 

Under  this  heading  come  all  general  stimuli  other  than  the 
scone  stimuli.  As  with  scope  stimuli,  the  verbal  stimuli  can  be 
classified  as  either  continuous  or  discrete. 

5.3*1  Continuous  Verbal  Stimulit  The  operators  in  the  CIC  bring 
with  them  a  general  background  on  both  the  enemy  and  friendly 
forces  derived  from  training  and  operational  experience.  This 
Includes  knov/ledge  of  the  characteristics  and  capabilities  of 
the  enemy’s  weapons;  the  speed  and  service  ceilings  for  the 


( 


enemy's  planes,  the  range  at  which  particular  weapons  can  usually 
be  detected,  and  their  characteristic  tactics  and  disposition. 

The  operator  has  the  same  sort  of  Information  on  his  own  forces: 
the  characteristics  of  friendly  planes,  their  usual  tactics  and 
disposition.  On  the  basis  of  these  facts,  he  Is  able  to  judge  the 
threat  of  particular  configurations  of  raids. 

An  operation  order  Is  provided  for  any  military  activity. 

This  written  order  Indicates  the  mission  of  the  airborne  CIC,  details 
specific  actions  to  be  taken  and  reports  to  be  made,  provides  Intel¬ 
ligence  on  enemy  activity  to  be  expected,  as  well  as  lnforv<ation  on 
the  mission,  disposition,  and  strength  of  friendly  forces.  All 
of  these  verbal  stimuli  qualify  the  system's  operation. 

The  continuous  verbal  stimuli  can  be  considered  an  element 
of  the  space  density  of  the  total  stimuli  to  the  system.  They  are 
not  particularly  manlpulatable  in  constructing  problems  of  dif¬ 
ferent  load.  It  Is  true  that  both  the  background  Information  and 
the  operation  order  furnished  to  the  operators  can  be  Increased 
in  complexity,  but  It  Is  extremely  difficult  to  scale  this  load. 

In  general,  the  continuous  verbal  stimuli  will  be  held  constant 
during  a  particular  experiment,  although  the  other  elements  of 
load  nay  be  varied.  The  complexity  of  the  continuous  verbal  stimuli 
has  been  gradually  increased  In  succeeding  system  studies  and 
probably  will  be  further  increased  In  studies  to  follow, 

5»3«2  Discrete  Verbal  Stimuli:  Included  in  this  category  are 
the  following: 

Specific  requests.  Instructions,  and  Information  from  the 
OTC  on  the  Command  radio  net 
Specific  requests  and  Information  from  other  Combat 

Information  Centers  on  the  Command  or  liaison  radio  nets 
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Specific  r<' quests  and  Inforr^atlon  from  friendly  forces 
(other  than  CAP)  on  liaison  or  VHP  radio  net's 

Specific  requests  and  Infornation  fron  GAP  (Combat  Air 
Patrol,  or  the  friendly  Intercepting  forces)  or  VHP 
radio  nets 

These  discrete  verbal  events  modify  the  nay  in  vhlch  the 
airborne  CIC  operates  and  determine  its  responses;  they  are  a 
port  of  the  time  density  load.  Here,  of  course,  the  combining 
of  events  into  a  time  density  score  must  be  most  carefully  done; 
it  c&n  be  readily  understood  that  the  performance  score  in  answer¬ 
ing  a  renuest  from  higher  authority  can  range  over  a  wide  scale. 
This  variable  is  manlpula table  and  r»ay  be  changed  rlthln  an 
experimental  run. 

In  considering  the  effects  of  these  various  stimuli  on 
the  system,  it  is  important  to  understand  which  links  of  the 
system  a  particular  kind  of  event  is  likely  to  stress.  For  the 
most  part,  the  verbal  stimuli  are  "stressful,"  at  least  immediately 
for  the  man  charged  with  integrating  the  system's  operation.  The 
scope  stimuli  are  components  of  load  for  the  radar  operators  or 
Air  Control  Officers.  Ultimately,  but  within  the  system,  the 
detailed  information  from  the  scope  picture  must  be  collected  and 
redisplayed  for  the  use  of  supervisory  and  ancillary  personnel. 

It  is  at  this  point  where  internal  scope  stimuli  become  Important 
for  the  effective  operation  of  other  links  in  the  system. 

5.3*3  Other  Discrete  Verbal  3timulit  Another  group  of  stimuli 
is  sometimes  presented  to  the  system.  For  instance,  the  CIC 
O'^flcer  might  be  informed  by  an  experimenter  (on  channels  other 
than  the  radio)  that  the  f.CO  4  Console  is  to  be  considered  to 
be  inoperative.  This  might  be  done  to  simulate  equipment  failure. 
Other  stimuli  of  the  same  sort  may  be  presented  to  the  system  for 
experimental  purposes. 
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The  nature  of  the  Input  to  the  system  has  been  dis¬ 
cussed;  in  this  section  the  anatomy  of  the  system  will  be  con¬ 
sidered.  The  way  In  which  the  system  responds  and  how  those 
responses  can  te  measured  will  follow  In  Section  7»0. 

Some  six  months  of  exploratory  work  went  on  In  the 
laboratory  prior  to  the  specific  consideration  of  system  variables. 
During  this  time  a  good  many  Insights  were  obtained  both  by  the 
crew  members  themselves,  who  had  prior  CIC  experience,  and  by 
the  experimenters,  whose  experience  both  In  CIC  work  and  Navy 
protocol  In  general  was  limited.  A  preliminary  listing  of  the 
variables  served  as  an  agenda  for  staff  conferences  demoted  to 
this  subject.  Over  a  period  o*'  some  five  days  discussions  of 
thr  operation  of  the  system  were  carried  on  In  which  Naval 
Officers,  experimental  psychologists.  Industrial  and  electronic 
engineers  participated.  These  conferences  were  aimed  at  a 
complete  consideration  of  the  system  and  the  possible  sources 
of  variation  In  system  operation  without  trying  to  develop  a 
set  of  categories  into  which  the  variables  might  fall.  Subsequent 
to  these  staff  conferences  the  variables  were  grouped  under  dif¬ 
ferent  headings  In  an  attempt  to  order  the  many  elements.  The 
listing  went  through  a  number  of  reviews,  and  Its  statement  In 
this  section  of  the  report  represents  considerable  digestion  and 
Integration. 

Analysis  of  the  system  Into  Its  possible  variables 
can  be  done  as  a  result  of  experience  with  the  airborne  CIC  system 
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and  v;lt)i  related  systems.  Actually,  the  list  that  follows  can 
scarcely  be  colled  ryperlmental  variables  In  the  usual  sense. 

It  Is  rather  an  Intimate  consideration  of  the  system  and  how  It 
works.  Nor  does  the  llstdnp  experimental  variables  Indicate  the 
way  In  which  answers  are  to  be  found.  As  a  matter  of  faet,  ex¬ 
perimental  work  bo^h  In  military  organizations  and  In  fundamental 
science  has  provided  answers  that  can  be  dravm  upon.  Some  of  the 
variables  must  be  studied  with  complete  operation  of  the  system; 
others  In  component  studies. 

The  discussions  that  follow,  as  mentioned  above.  Indicate 
sources  of  variation  In  system  operation  without  the  explicit 
statement  of  reasonable  alternatives.  It  is  felt  that  these 
"reasonable  alternative”  methods  of  operation  will  emerge  as  ex¬ 
perimental  work  proceeds.  In  reviewing  the  list  of  variables  it 
has  been  apparent  that  some  procedures  are  more  critical  and 
important  for  study  at  this  time.  It  Is  believed  that  In  ap¬ 
proaching  the  experimental  work  In  this  fashion  the  more  critical 
problems  to  study  will  develop  one  by  one  together  with  the  logical 
alternative  methods  that  might  be  employed.  Nor  does  the  listing 
necessarily  imply  that  there  are  no  available  answers  to  the 
particular  questions — indeed  both  compelling  operational  ex¬ 
perience  and  psychological  data  may  indicate  the  appropriate  solution 
without  additional  work. 

The  variables  in  the  airborne  CIC  can  be  classified  under 
three  general  headings! 

(1)  Behavioral  or  procedural  variables 

(2)  General  or  e  vironmental  conditions  under  which 
the  system  operates  (In  this  category  fall  problems 
In  criteria  of  effective  operation) 

(3)  Design  variables 
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F^rh  of  those  three  ponoral  catoporlos  Is  nenlpula table 
to  sone  extent:  that  is,  the  oporatinp  procedures  for  the  system 
have  not  os  yet  bo^  stabllshed,  and  appropriate  manners  of 
working  eon  be  recommended:  there  is  freedom  to  recommend  the 
environmental  conditions  which  should  prevail  for  most  effective 
system  operation;  and  finally  there  are  a  good  many  aspects  oi*  the 
design  of  the  equ‘'pment  Itself  and  the  establishment  of  the  links 
that  should  exist  in  the  systcT"  which  are  quite  Independent  of 
technical  Issues. 


6.2  List  and  Descrintion  of  Variables 

6.2.1  Behavioral  Variables:  The  behavioral  variables  can 
be  categorized  under  several  headings: 


6. 2. 1.1  Procedures  Governing  the  Scope  Activity 
of  One  AGO 


There  are  a  number  of  ele’^ents  contributing 
to  the  v’ay  in  which  the  AGO  orients  his  scope 
picture.  Should  he  use  a  grid  or  should  he 
use  nolar  coordinates  comprised  of  angle  and 
range  marks?  If  he  is  to  use  a  rectangular 
grid,  whrt  should  be  its  relative  brightness 
to  the  radar  pips,  the  width  of  line,  the 
scale  and  th^-  numbering  teclmique  under  the 
four  range  conditions  with  which  he  may  operate? 
Chould  he  work  v'lth  his  sweep  centered  or 
should  be  off cent or  his  sweep  in  order  to 
search  particular  sectors?  Inasmuch  as  the 
operator  is  in  a  plane  which  moves  very 
rapidly  with  respect  to  the  ground,  should  he 
Ground  stabilization  work  from  a  radar  nicture  that  is  ground  stabi- 
and  grid  positioning  lized?  (This  facility  is  provided  In  the 
correc+'lons  P0-2W  and  in  effect  roves  the  svreep  over  his 

scope  face.)  The  navigator  has  at  his  disposal 
controls  which  enable  •him  to  correct  the  grid 
position  as  the  result  of  his  navigational 
fixes.  VJhc.t  diffieulty  does  this  pose  for  the 
radar  operator  and  his  tracks?  Correction  will 
be  disturbing  to  the  tracks,  and  the  question 
is  how  can  the  corrections  best  be  incorporated 
into  the  operator's  activity,  and  at  what  time? 


Locating  Procedure 

Grid  or  polar 
coordlna  tes 

Type  of  grid 

Centered  or  off- 
centpred  scope 
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Search  Procedure 
Optimum  illumina¬ 
tion 

Grid  on  or  off 

Scanning  habits 

Estimating  Pro¬ 
cedure 

Plot^-lng  tools 
Use  of  desk  board 


6. 2. 1.2 

Communication 
ICS 

Video  insertion 


Division  of  V/ork 


Collective  or 
divided  search 
Scope  division  by 
azimuth  or  range 

Sector  "ize 
Scope  divijion 
by  function 

Number  of  scopes 


6. 2. 1.3 

Height  Finder 
Priority 


Navigator- ground 
stabilization 


During  search  activities  rhat  illumination 
levels  are  appropriate  for  the  scope  in 
order  to  get  best  contrast  in  performance? 

Should  the  grid  (either  polar  or  rectangular) 
be  "on"  or  "off"  during  search  phases?  What 
scanning  habits  should  the  operator  adopt? 

In  estimating  courses  and  speeds,  v;hat 
plotting  tools  and  computational  aids  are 
necessary?  How  best  can  the  operator's  desk 
board  be  used  for  storing  information?  In 
fact,  v’hat  kind  of  information  is  it  necessary 
for  the  operator  to  "store"? 

Between  ACOs 

One  aspect  of  coordination  between  ACOs  is 
that  of  communication  technique.  Should  the 
ICS  be  used  exclusively  or  does  video 
Insertion  have  a  useful  function  in  this 
procedure?  What  kinds  of  information  need  to 
be  passed  or  can  be  best  transmitted  by  each 
of  these  means?  What  ICS  procedure  best  ac¬ 
complishes  inter-ACO  communication? 

Another  question  of  resolving  inter-ACO 
actlvi^-y  is  that  of  division  of  work.  Should 
the  ACOs  divide  the  radar  surveillance  area 
into  sectors,  or  should  each  ACO  search  the 
entire  area?  If  the  scope  is  to  be  divided  into 
sectors,  should  that  division  be  by  azimuth 
or  range  sectors?  Should  the  division  of  the 
scope  be  into  equal  sized  sectors,  or  sectors 
determined  by  extent  of  activity?  v'^hould  the 
work  of  the  ACOs  be  divided  by  function  rather 
than  by  area;  that  is,  should  some  ACOs  only 
search  for  new  targets  while  others  track  the 
targets  that  have  been  detected?  How  many 
scopes  are  necessary  in  the  system  in  order  to 
achieve  the  desired  per'^ormance  under  various 
load  conditions? 

Between  ACOs  and  Other  Information  Sources 

If  the  ACOs  are  going  to  request  altitude 
information  from  the  height  finder,  which  of 
the  two  priority  systems  should  be  used 
(automatic  or  manual)? 

Another  source  of  information  for  the  ACO  is 
the  navigator.  What  coordination  is  neces¬ 
sary  between  mCOs  and  the  navigator  in  the 
process  of  correcting  the  ground-stabilized 
grid? 


Assistant  CIC 


Conmunlcation-ICS 
or  Video  Link 


Target  designation 
6. 2. 1.4 

Charnel 

ICS  to  local 
display 
Radio  to  OTC 
PO  link 


Procedure 
Round  robin 

Selective 

sv/ltchlng 


6.2,1. 5 

Methods. 


Source  of  Request 
AGO 

CIC  Officer 


Local  display 


The  Assistant  CIC  Officer's  functions 
lnr']ude  coordinating  the  activity  of  the  ACOs, 
In  this  connection  ’"hat  kinds  of  infor’^’itlon 
should  the  Assistant  CIC  Officer  furnish  the 
ACOs?  To  what  extent  should  he  coordinate 
their  efforts?  Should  he  rely  on  the  inter¬ 
communication  system  or  the  video  link  for 
transmitting  particular  kinds  of  Information? 
Should  the  Assistant  CIC  Officer  designate 
new  targets  and  control  Initial  reports? 

Between  ACOs  and  Information  Destination 

In  communicating  the  information  which  he 
reads  off  his  scope  should  the  ACO  convey  It 
by  ^ho  ICS  to  a  local  display,  or  should  he 
transmit  It  by  radio  directly  to  the  OTC? 

Or  should  this  Inforr^atlon  be  transmitted  by 
the  PO  link  In  part?  The  question  Is  that 
of  direct  vs.  Indirect  reporting. 

In  effecting  best  communication  of  information, 
should  the  ACOs  maintain  a  "round  robin" 
talking  procedure,  which  means  that  all  the 
ACOs  listen  to  each  other's  transmissions  at 
all  times,  or  should  each  ACO  transmit 
individually,  either  to  the  local  display 
or  via  the  radio  to  OTC,  only  when  cued  by 
someone  else  in  the  team?  This  latter  pro¬ 
cedure  would  mean  that  the  ACO  would  not  be 
burdened  with  listening  to  Information  which 
might  be  extraneous  to  his  particular  job  and 
would  only  have  to  listen  on  the  communication 
channel  when  It  v;as  his  turn  to  m.ake  a  report. 
It  would  enable  him  to  engage  In  cross-com¬ 
munication  with  other  ACOs  and  to  perform  his 
various  estimates  without  concerning  himself 
with  other  Information, 

Height  Finder 

The  height  finder  operator  has  several  ways 
In  which  he  can  operate.  He  can  use  his  scope 
both  as  a  PPI  and  as  a  height  finder.  He 
can  either  survey  the  entire  area,  finding 
altitudes  on  targets  on  his  ov/n  initiative,  or 
respond  to  requests  for  information  about 
particular  targets.  V/hat  should  be  the  source 
for  the  request  for  altitude  information? 

Should  the  ACOs  inltiat'®  all  requests  for 
height  Information;  should  the  CIC  Of'f'lcer  or 
his  Assistant  initiate  requests,  or  should  the 
height  finder  operator  find  altitudes  on  the 
be  sis  of  current  information  which  he  sees  in 
the  local  display? 
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Outlot 


AGO 

Local  display 
CIC  Officer 

Alternative  ?!ethods 


6. 2. 1.6 

Integration  and 
tlai.^on 


ACOs 

Local  display 

ACICO 

Height  finder 

ECI^  operator 
Radio  &  radar 
operators 
Pilot  &  Navigator 


/.Itcrnatlve  methods  of  op'^ratlon  of  the  height 
finder  operator  can  also  be  categorized  vlth 
respect  to  the  outlet  of  f^ls  altitude  Informa¬ 
tion.  Should  he  transmit  the  altitude  on  a 
particular  target  to  the  AGO  requesting  It  by 
means  of  the  data  transmission  system  only, 
or  should  he  give  that  Information  to  the  lUCO 
over  the  ICS?  Should  he  transmit  altitude 
Information  to  the  local  display  by  means 
of  ihe  ICS  or  to  the  GIG  Officer  either  by  ICS 
or  In  other  fashions? 

There  are  alternative  methods  by  rhlch  the 
height  finder  actually  .manipulates  his  gear. 

He  Is  eued  rh^'n  requests  for  Information  come 
through  the  priority  system  by  having  the 
height  finder  antenna  slewed  Into  the  azimuth 
area  of  the  target  on  which  Information  Is 
desired,  il^  range  strobe  also  appears  on  his 
display  which  Indicates  the  range  of  the 
target  In  question.  There  are,  of  course, 
occasions  Ir  which  several  targets  appear  In 
the  same  area  which  make  it  difficult  for  him 
to  discriminate  on  the  basis  of  the  strobe 
(which  may  be  somewhat  off  In  range)  without 
additional  Information  from  the  requestor. 
Other  questions  invol^'lng  method  of  operation 
of  the  same  sort  probably  exist. 

The  CIC  Officer 

The  CIC  Officer  not  only  supervises  and 
integrates  the  activity  of  the  Combat  Informa¬ 
tion  Center,  but  is  also  the  one  man  who  per¬ 
forms  certain  functions.  Hov'  does  he  maintain 
liaison  with  the  members  of  his  team  within 
the  plane  and  integrate  their  activity?  On 
the  basis  of  information  which  comes  to  him 
from  outside  sources  it  may  be  necessary  for 
him  to  modify  ACOs’  reporting  procedures. 

How  does  hr-  commimlcate  this  to  the  ACOs? 
Should  he  use  the  ICS  or  should  he  move  next 
to  the  man  In  question  and  communicate  the 
Information  orally?  Should  he  Insert  Informa¬ 
tion  on  the  central  display  board  from  which 
the  ACOs  will  get  their  cues?  What  are  the 
appropriate  kinds  of  Information  to  flow 
between  the  CIC  Officer  and  his  assistant, 
and  what  vehicle  should  be  used?  Should  the 
CIC  Officer  administer  the  height  finder 
priority  system,  or  should  that  function  be 
delegated  to  his  assistants?  What  kinds  of 
communication  flow  between  the  CIC  Cficer 
and  ancillary  personnel  such  as  the  ECl'f 
operator,  the  radio  onerator,  and  the  radar 
operator?  V/hat  kinds  of  coordinating  function 
prevail  between  the  CIC  Officer  and  the  plane 
crew  such  as  the  pilot  and  the  navigator? 
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Contact  v;ltb  PTC 


In  rolntai*' jnp  syster  r*ontact  vjth  OTC, 
to  rliat  extent  phouild  tbo  CIC  Officer  pprtlcl 
pote?  Should  he  request  oil  In^’ornatlon  fror 
Evaluation  and  thr  OTC?  Should  hs*  do  all  evaluation  and 

tactical  comment  make  all  tactical  comments?  fhould  these 

evaluations  or  tactical  eo'^ments  he  made 
only  upon  request,  or  spontaneously  as  situ¬ 
ations  develop? 


Contact  V7ith  Others 
CAP 

Position 
It  scope 

Standing 

At  desk 


To  rhot  extent  Is  it  necessary  for  the  CIC 
Officer  to  supr-rvise  intercepts,  communicate 
rith  CAP,  and  have  immediate  physical  sur¬ 
veillance  of  the  work  of  his  team  members? 

In  order  to  best  serve  his  ^unctions  should 
the  CIC  Officer  actually  work  at  a  radar  scope, 
maintaining  tracks,  or  should  he  be  free  to 
move  around?  fhouid  he  stand  where  he  can 
watch  a  radar  scope  which  is  being  plotted; 
should  he  have  access  to  the  airborne  DRT, 
verbal  displays  or  geographical  plots?  Or 
should  he  work  at  a  desk  where  he  can  best 
accomplish  the  paper  work  that  is  necessary 
for  carrying  out  the  operation  order? 


6. 2. 1.7  Local  Displays 


las 

Kinds  of  status 
hoard 


Geographical  plot 
DRT 


Position 


ViHiat  typos  of  local  displays  are  necessary, 
and  to  v;hom  should  they  be  available?  ’.Vhat 
kinds  of  verbal  displays  or  status  boards  are 
necessary?  Should  there  be  several  types  of 
status  boards — for  instance,  one  with  radio 
calls,  one  I'-ith  information  on  surface  targets, 
one  for  air  targ'-ts,  one  for  "hot"  information 
as  against  '^ore  permanent  information?  Should 
th«re  be  a  vertical  or  a  horizontal  geographi¬ 
cal  plot?  V«'ill  thf  airborne  DRT  be  sufficient? 
Of  course,  it  may  be  that  a  combination  of 
these  display  boards  is  necessary,  and  it  is 
then  important  to  determine  the  most  advan¬ 
tageous  position  for  each  of  them. 


Content 


Gra phlc 
Verbal 


A  question  that  is  very  closely  correlated 
with  the  type  of  display  needed  is  that  of 
appropriate  content.  What  kinds  of  Information 
need  to  be  contained  in  each  of  the  selected 
displays?  In  the  case  of  graphic  representa¬ 
tion,  how  should  the  information  be  coded? 

In  the  case  of  verbal  displays,  what  tabular 
form  is  most  appropriate? 

6. 2. 1,8  T'fiscellaneous 


Under  this  "catch  all"  heading  are  a  number 
of  qu'^stlons  which  are  not  easily  classifiable, 
Kow  is  information  from  sources  other  than 
those  listed  obtained  and  integrated?  How  can 
the  video  insertion  unit  be  best  used  pro¬ 
cedure-wise,  and  whrt  kinds  of  information  best 
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Video  insertion 


EC?ir 

Radarnan 
CTV  op ora tor 


l^nd  thf‘r's«»l'’^os  to  thjs  typ.?  of  transnl  sslon. 
Hov  are  thr^  activities  of  the  ECI’  operator, 
♦.he  radio  ncn,  the  radar  »"an,  and  the 
electronics  technician  best  Inteprated  Into 
this  system?  What  kinds  of  Information  does 
the  CIC  Officer  need  from  these  crev  nembers, 
and  vihi-t  kinds  of  direction  do  they  require 
from  him? 


Camera 

Information  from 
other  sources 


Hov  can  the  camera  unit  be  best  used?  How 
should  the  PO  link  be  administered ,  and  v-hat 
types  of  Information  can  uest  be  passed  by 
this  TTK.'ans?  Hov'  can  the  Incoming  PO  link 
from  other  AEIV  pianos  b«  used?  Should  the  CIC 
Officer  ’"otch  this  display,  and  how  can  Iv  best 
insert  this  sort  of  Information  Into  the 
system's  operation? 


6.2.2  General  or  Environmental  Variables:  The  general 


variables  can  be  classified  Into  four  groups: 


(11  Nature  of  the  Mission 

(2)  Environmental  Factors 

(3)  Personnel 
(41  Contingencies 


All  of  these  factors  have  to  do  with  the  milieu 


v’ithin  ’■■’hich  the  system  operates.  These  are  the  exterior 
forces  that  rsteblish  the  environment. 

6. 2. 2.1  Mission 


AETr  only 
With  intercept 
Other  functions 


The  P0-2r  aircraft  can  and  probably  will 
serve  a  number  of  functions  in  any  operational 
condition.  It  may  serve  such  functions  as 
AE7.’,  iSV’,  amphibious  landing  coordination  or 
alr/sea  rescue,  each  separately  on  one 
mission  or  several  functions  simultaneously 
within  one  mission.  Each  of  these  missions 
will  Influence  the  kind  of  output  desired 
from  the  system. 


Information  tp  PTC 

Type 

Amount 


Even  when  the  mission  has  been  established, 
the  precise  way  in  which  it  is  to  be  carried 
out  will  he  subject  to  the  desires  of  the 
OTC  and  certain  operational  condi^’lons.  The 
system  will  require  different  procedures, 
and  possibly  different  links,  if  it  is  to 
furnish  large  quantities  of  detailed  informa¬ 
tion  rather  than  periodic  general  summaries 
of  the  situation. 


Those  questions  ltg  at  the  core  of  the 


problem  of  oritorlon  of  effective  perfor’^ance.  Boasonable 


alternative  netliods  of  operation  can  be  found  only  by  a  con¬ 
sideration  of  the  larger  system  into  v;hich  the  airborne  CIC 
fits.  Operation  research  methods  can  help  determine  v-hat  types 
of  information  are  essential  for  this  element  o;'’  the  entire 
system  to  perform. 

6. 2. 2. 2  Environmental  Factors 


Weather 


Illumination 


Noise 

Engine 


Radio 


The  general  flying  (Conditions  ^-^hich  prevail 
will  probably  substantially  affect  crew 
comfort  and  nay  under  extreme  conditions 
thoroughly  disrupt  both  the  physical 
operation  of  the  equipment  and  the  ability 
of  the  human  operators  to  perform. 

In  a  CIC  different  kinds  of  illumination 
are  r^-quired  at  different  points.  Perhaps 
in  the  past  CIC s  have  erred  in  the  direction 
of  beeping  the  Illumination  level  too  low. 

The  problem  here  is  to  reach  ef'''ective  com¬ 
promises  xvhich  will  permit  optimal  scope- 
watching  activities  mpiie  pro-'^iding  suf¬ 
ficient  general  illumination  for  other 
operators  to  perform  their  work  conveniently 
and  wi+-hout  error. 

The  noise  level  that  prevails  within  the 
airplane,  primarily  because  of  the  airplane's 
engines,  effectively  reduces  the  possibility 
of  oral  communi^i  tlon  v/lthout  the  benefit 
of  earphones,  ’'.'hether  this  effect  contributes 
substantially  to  operator  fatigue  and  ability 
to  perform  is  an  open  question.  The  radio 
noise,  of  course,  reduces  the  intelligibility 
of  the  conmunleatlons  and  may  also  have  the 
overall  effect  of  inducing  operator  fatigue 
and  error. 


Tempera ture  The  temperature  and  humidity  prevailing  in 

the  work  area  are  very  important  for  operator 
comfort.  Previous  studies  ha-^e  indicated  that 
unfavorable  temperature  and  humidity  conditions 
adversel;^  affect  operator  performance  and 
stamina  • 

Vibration  Vibration,  in  contrast  to  pitch  or  roll  of 

the  rlane  due  to  v/eathrr,  v'ill  probably  also 
contribute  to  operator  fatigue. 


i 


■1 
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So^e  of  there  en'''ironmental  oonclltions 


are  readily  nanipula table ,  and  speoi''"i^ations  should  he  drawn 
up  which  will  establish  optimal  v'orklng  ro’-ditions  for  the 
operators.  Experimental  investigr t' ons  will  show  the  specific 


amount  of  performance  decrement  aecruing  because  of  adve’^se 
eonditions.  These  data  v/ill  demonstrate  how  important  it  will 
be  to  modify  existing  conditions, 

6.2.2, 3  Personnel 


Motivation 


Operational 

realism 


Learning 


Fatigue 


A  matter  of  great  concern  in  any  investi¬ 
gation  of  human  behavior  is  that  factor 
termed  motivation.  It  has  been  demonstrated 
both  under  experimental  and  operational 
conditions  that  an  operator  who  is  highly 
motivated  can  maintain  high  efficiency  for 
long  periods  of  time  under  adverse  conditions. 
Average  human  beings  do  remarkable  feats 
which  even  they  did  not  consider  themselves 
capable  of  under  stress  conditions.  It  is 
questionable  whether  any  laboratory  study 
can  duplicate  the  high  motivation  existing 
under  an  operational  or  v>artime  condition. 

Of  course,  operators  do  not  always  perform 
"over  their  heads";  very  good  operators 
sometimes  break  down  completely  under  stress. 
The  experimenter  does  not  usually  have  at 
his  disposal  any  means  to  vary  this  factor 
over  a  significant  range  for  experimental 
subjects. 

During  any  performance  an  operator  learns. 
Sometimes  he  shoves  remarkable  improvement 
from  day  to  day;  at  other  times  his  effective¬ 
ness  seems  to  have  reached  a  "leveling  off." 

In  laboratory  experiments  different  operating 
procedures  could  be  compared  as  to  the  ease 
with  which  they  could  be  learned  by  a  crew. 

The  other  approach  which  is  used  in  these 
investigations  is  to  try  to  orient  the  crew 
sufficiently  in  a  particular  procedure  so  that 
they  have  reached  a  "leveling  off"  in 
proficiency.  This  latter  procedure  provides 
a  comparison  of  different  operating  conditions 
under  more  stable  conditions. 

The  human  operator,  of  course,  gets  tired,  and 
as  he  porfor»^s  over  a  period  of  time  his  ef¬ 
fectiveness  is  reduced.  There  are  many  factors 
which  contribute  to  fatigue,  and  a  systematic 
exploration  of  fatigue  against  performance 
can  be  made. 
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Rotation  Any  operational  condition  involves  a  great 

deal  of  boring  waiting  for  sonething  to 
Duty  VVrtches  hanren.  This  substantially  affects  operator 

perfor’^ance.  It  i^ay  be  desirable  to  explore 
the  effect  of  system  performance  under 
conditions  where  nothing  happens  for  long 
periods  of  time.  It  also  may  bo  worthivhlle 
to  study  how  frequently  operators  should  be 
rotated* from  job  to  Job  under  load  conditions, 

6.2,r.4  Contingencies 

I''ost  r^issions  do  not  run  as  planned;  something 
always  seems  to  happen.  Equipment  may  fall  at  a  critical  time; 
there  may  be  a  human  disability  resulting  from  natural  conditions 
or  enemy  action,  or  it  nay  be  necessary  to  shift  procedures  to 
meet  a  nev;  threat.  Each  one  of  these  contingencies  Is  a  shock 
to  the  smooth-running  system  and  may  disrupt  operations  completely. 
It  will  be  possible  to  study  in  the  la^'orrtory  the  effects  of 
these  various  contingencies  on  system  operation. 

6.2,3  Deslpn  Variables:  The  design  variables  are  those 
that  change  the  physical  aspects  of  the  equipment  In  the  system 
In  accordance  v/lth  human  engineering  principles.  Also  included 
arc  those  in  which  links  are  added  to  or  taken  from  the  system. 

For  exar^ple,  adding  a  new  display  v/lth  the  personnel  necessary 
to  maintain  that  display  would  be  considered  a  modification  of 
the  systems  design. 

One  of  the  major  questions  in  the  human  engineering 
design  of  a  system  is  the  arrangement  of  the  main  items  of 
equipment  within  the  space  allowed. 

An  example  of  the  positioning  of  subunits  within 
each  major  station,  for  instance,  is  the  proper  placing  of  the 
ICS  box  at  an  ACO’s  station.  Another  problem  is  the  design 
of  auxiliary  equipment  such  as  crew  seats.  Such  an  area  is 

•t-ermed  component  study  where  the  operator  is  considered  in 
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relation  to  the  dials  that  he  must  see  and  the  controls  that 
he  must  use.  The  coi^ponent  must  be  designed  to  facilitate 
operator  performance. 

The  link  studies  are  also  tied  up  with  the  behavioral 
variable  because  an  operating  procedure  is  necessarily  influenced 
by  the  links  available. 


7.0  THE  PR0BLEI<!  OF  rTijArURFJOT 


In  previous  sections  the  nature  of  the  stinulus  and  the 
organism  have  been  discussed.  In  this  section  the  typos  of  response 
of  the  system  and  the  ways  in  which  those  can  be  r^easured  vrlll  be 
considered  together  with  ways  of  describing  internal  system  activity. 

The  system's  output,  v;hich  will  vary  with  the  kind  of  mission 
that  Is  performed,  could  Include  the  following: 

(1)  Reports  on  the  command  channel  which  are  substantially 
those  to  the  OTC. 

(2)  Reports  on  the  liaison  channel  to  other  CICs  or 
friendly  forces. 

(3)  Reports  on  the  five  VHF  transmitters  (v-hich  nay  be  on 
any  of  some  ten  channels)  which  send  information  to  CAP. 

(4)  PO  link  transmissions  to  the  OTC  or  to  other  CICs. 

(5)  System  actions  which  cause  the  flight  pattern  of  the 
AE7/  plane  itself  to  be  modified. 

The  problem  of  measurement  will  make  it  necessary  to  describe 
the  nature  of  data  which  can  be  collected,  indicate  the  kinds  of 
responses  that  the  systems  can  make,  categorize  the  measures  of 
systems'  outputs  and  review  the  possible  Internal  measures. 

It  is  Important  to  state  that  the  measures  to  be  used  must 
be  efficient.  V-Tille  it  may  be  possible  vrlth  a  great  deal  of  trouble 
to  get  very  precise  measures  requiring  considerable  expense  and  time 
to  collect  and  analyze,  it  is  more  Important  to  get  data  in  such  a 
fashion  and  to  use  measures  of  such  a  kind  that  analysis  can  nroceed 
very  rapidly.  It  should  be  emphasized  that  molar  units  are  needed 
in  applied  research. 

7.1  Nature  of  the  Data 

Although  the  data  which  are  collected  during  an  experiment 
can  be  categorized  into  (a)  thO':e  which  pertain  to  output,  (b) 


thofso  which  are  checks  on  Input,  and  (c)  those  which  ere  checks 
on  hov  precisely  the  orpanlsm  -^unctions  according  to  the  experl- 
nontal  design,  the  rea cures  do  h6'«'e  ures  for  other  purposes  than 
those  Into  which  they  have  been  classified.  In  other  words,  the 
categories  are  not  mutually  exclusive. 

(1)  The  output  measures  are  obtained  from  tape  reeordlngs 
of  activity  on  the  four  radio  channels  r^entloned  abo^^e,  and  on 
the  flight  crew  ICS. 

(2)  The  following  records  are  used  to  check  on  the  Input 
to  the  system:  the  tracing  paper  plot  of  the  radar  picture  as 

It  appears  at  the  VG,  the  camera  recordings  of  the  radar  picture 
on  the  ArA-56  equipment,  and  the  script  followed  by  the  simulator 
operators. 

(3)  The  way  In  wh" ch  the  system  Itself  operates  can  be 
checked  by  data  gathered  from:  reports  of  the  crew  members 
following  each  experl’^ent,  tape  recordings  of  the  activity  of 
the  Intercommunication  system,  and  observations  of  the  activity 
of  the  various  crew  mem.bers  during  a  problem.  This  last  source 
of  information  may  be  obtained  by  time-study  techniques  or  micro¬ 
motion  analyses. 

7.2  Responses 

The  stimuli  to  the  system  have  been  described  in  detail; 
the  resonses  that  the  system  can  make  co  the  stimuli  are  equally 
varied  and  can  be  gro’  ped  into  single  and  multiple  responses. 
7.2.1  Single  Responses  to  fltlmull:  Included  in  this  category 
are  responses  to  thr  discrete  scope  and  verbal  stimuli  plus  those 
responses  to  continuous  scope  and  ver'''fl  stimuli  which,  once  made 
need  not  be  repeated  unless  there  Is  a  change. 


The  resp-Tises  to  discrete  scope  and  ^^erbal  stlnull 
include: 

New  targets 

Fedes  and  reappearances 
IFF  signal 
Change  in  course 
Change  in  speed 
Change  in  altitude 

Motion  of  target  due  to  entering  weather  area 

Leaving  the  area  of  surveillance 

Entering  sea  return 

Certain  reports  specifically  requested 

Those  single  respenses  to  continuous  scope  and  verbal 

stimuli  Include: 

Initial  course  on  target 
Initial  speed  .. 

Initial  altitude 
Composition 
Initial  vector  to  CAP 
Turn  vector  to  GAP 
Homing  Information  to  CAP 

Certain  reports  on  the  tactical  situation  called 
for  by  the  Oporder 

7.2.2  Multinle  Responses  to  Stimuli;  In  this  group  of  responses 
are  those  made  to  continuous  scope  and  verbal  stimuli  as  follows: 
Position  reports 

Reports  to  pilot  which  modify  plane  track 
Vectors  to  GAP  (other  than  initial  and  turn) 

Certain  tactical  reports  called  for  by  the  Oporder 
Certain  reports  established  by  specific  request 

7.3  Output  I'^easurement s 

The  system  performance  can  be  measured  in  three  ways:  by 
production  rate,  latency,  and  accuracy  of  the  output.  These 
three  basic  aspects  of  output  may  very  in  importance  depending 
on  the  mission  of  the  AEV/  plane.  The  problem  becomes  that  of 
determining  the  relative  importance  of  these  three  elements  in 
systems  output.  For  example,  it  may  be  found  that  a  certain 
majl:imal  error  in  accuracy  of  position  reports  can  be  tolerated 
in  certain  types  of  missions — any  improvement  in  accuracy  would 


not  improve  the  performance  of  the  system.  It  may  also  he 
found  that  the  initial  vector  to  a  CAP  requires  the  shortest 
latency  hut  very  little  accuracy. 

Time  and  space  density,  together  v'ith  the  time  at  which 
the  stimulus  occurs  during  a  work  period,  have  been  suggested 
as  elements  of  the  stimulus  load.  Besides  comparing  the 
system's  performance  under  different  operating  conditions,  it  is 
essential  to  discover  the  function  between  the  various  outputs 
and  the  elements  of  the  stimulus  load.  It  will  be  noted  that 
a  number  of  measures  suggested  are  so  orianted. 

7.3»1  Production  I^feasures: 

The  production  rates  are  perhaps  the  easiest  to  measure. 

The  measures  listed  below  become  nrogressively  more  specific; 
they  are  based  on  a  number  of  reports  made  per  unit  of  time. 

This  unit  of  time  may  be  a  minute  or  longer,  depending  on  the 
homogeneity  of  the  stimulus  during  the  time  unit.  The  time  unit 
should  be  selected  so  that  the  analysis  rill  compare  homogeneous 
units. 

7.3»1»1  Total  Number  of  Reports  Fade  by  the  System  per  Unit  Time 

This  is  a  gross  measure  which  may  reveal  the  characteristic 
response  rate  for  the  system  over  a  range  of  load 
conditions.  That  is,  the  system  may  continue  to  make  the 
sa’^e  number  of  position  rejjorts  per  minute  with  a  load  of 
six  bogies  as  v/lth  12  bogies,  simply  making  twice  as  many 
under  the  ■f’lrst  condition.  As  a  characteristic  resnonse 
rate,  it  may  be  both  a  gross  indicator  of  the  effective¬ 
ness  of  a  particular,  configuration  for  the  system  and  of 
the  effect  of  continued  operation  on  output. 

7.3»1»2  Number  of  Reports  per  Channel  per  Unit  Time 

The  measure  of  the  number  of  reports  made  on  the  command 
channel,  liaison  channel,  and  on  the  VHF  channels, 
will  be  much  affected  by  the  nature  of  the  mission  as¬ 
signed  to  the  AE\V  Plane. 
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7.3»1»3  Number  of  Reports  per  Type  per  Unit  Time 

By  type  Is  meant  those  responses  Iter^ized  In  Section  7»2, 

The  hulk  of  reports  vlll  probably  be  position  reports. 

7.3»1»^  Number  of  Reports  per  Target  per  Nlnute 

The  measures  Indicatpd  above  do  not  tak^  •’ntc  cc''cunt  ho;v 
adequately  the  system  provides  reports  on  all  of  the  tar¬ 
gets  v/lth  which  it  has  to  deal.  The  system  could  maintain 
a  high  output  rate  and  not  report  the  position  of  Target 
las  frequently  as  Target  2.  This  measure  Is  an  attempt 
to  see  how  uniformly  each  of  the  targets  is  reported  when 
no  priority  on  them  is  used.  It  way  serve  as  well  as  an 
indicator  cf  how  effectively  any  priority  system  works. 

The  Ideal  system  would  adapt  itself  to  report  more  fre¬ 
quently  those  targets  which  represent  a  greater  threat. 
V/hether  a  significantly  higher  report  return  is  maintained 
on  the  threatening  targets  can  be  determined. 

Also  included  are  those  reports  made  to  CAP.  It  would  be 
a  matter  of  Interest  to  discover  whether  the  success  of  an 
Intercept  depends  on  the  frequency  o:^  "sectors  to  CAP. 

7.3»1»5  Transmission  rates 

This  measure  proposes  to  explore  the  efficient  use  of  radio 
time.  It  Is,  of  course,  affected  both  by  the  amount  of 
"dead"  time  and  the  rate  at  v'hich  information  is  talked. 
There  is  some  dif^l^ulty  in  determining  a  useful  unit  for 
measuring  information,  ^'Ut  by  measuring  dead  time  or  the 
duration  of  actual  transmissions  an  estimate  can  be  made 
of  how  effectively  air  time  is  used.  It  will  allow  an 
estimate  of  the  ideal  information  load  in  a  period  of 
time,  with  a  later  determination  of  how  efficiently  any 
form  of  the  system  uses  this  capacity. 

7.3*2  Latency  I'^easures: 

*■  Latency  is  a  less  available  measure.  The  transcript  of  the 

responses  of  the  system  Is  all  that  is  reouired  to  get  the  production 
data;  however,  for  latency  measures  it  is  necessary  to  have  as  a 
reference  the  time  at  which  the  stimulus  occurred  to  find  the 

1 

latency  of  the  response.  Three  means  of  providing  this  reference 
point  have  been  used: 

1)  A  film  record  of  tlie  hapnenlngs  on  the  scope  face 

2)  The  script 

3)  A  signal  imposed  on  the  tape  recording  of  the  system 
output  indicating  the  time  at  which  the  stimulus 
occurred. 
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The  first  technique  mentioned  js  the  most  laborious.  The 

other  tv;o  have  their  disadvantages  as  veil,  and  the  value  of  the 

respeotive  techniques  can  be  determined  finally  only  from  more 

extensive  experience. 

7.3»2.1  Latency  of  Report  by  Type 

There  v'ill  be  substantial  differences  in  the  latency 
of  reports  as  tabulated  in  Section  7.2.  For  some 
latency  should  be  measured  from  the  time  of  the 
stimulus;  in  others  the  latency  should  be  measured 
from  •'■he  time  of  a  previous  response.  Latency  of 
reports  of  new  targets  and  loss  of  targets  should  be 
measured  from  the  time  of  stimulus.  The  latency  of  the 
initial  course,  speed  and  altitude  reports  should  more 
properly  be  measured  from  the  report  of  the  first  ap¬ 
pearance  of  that  target. 

7. 3 *2. 2  Latency  of  Report  by  Target  or  Group  of  Targets 

This  is  a  finer  measure  vhlch  serves  the  same  purpose 
as  does  that  in  7»3«1«4-  in  indicating  how  well  a  prior- 
itv  system  is  providing  quicker  reports  on  more  threaten¬ 
ing  targets. 

7.3*3  Accuracy: 

A  criterion  of  accuracy  is  even  more  difficult  to  establish 
than  that  for  latency.  In  any  study  of  performance  of  mechanisms 
operated  by  man,  it  is  difficult  to  distinguish  machine  from  operator 
error.  Errors  themselves  can  be  either  of  the  constant  or  random 
type.  The  random  error  is  a  lack  of  consistency;  the  constant  error 
is  a  result  consistently  at  variance  from  the  true.  For  exan^ple, 
if  a  rifleman  is  shooting  a  tight  group  into  the  target  but  off 
hull's  eye,  he  can  remedy  this  rather  readily.  If  another  rifleman 
is  scattering  Ms  shots  all  over  the  target,  he  is  faced  with  more 
trouble  in  correcting.  The  first  is  an  example  of  a  constant  error, 
the  second  of  random  error. 

Constant  errors  in  the  radar  gear  are  probably  of  greater 


magnitude  thfn  random  errors,  t-ypprience  v/ould  indicate  that  mhlle 

-  73  - 


the  sl’~ulr.ted  tarpet  r;ill  not  necess&rily  follow  the  desipniited 
course  exactly,  it  will  follow  a  stralpht-line  course.  While 
the  operator's  performance  could  have  both  constant  and  random 
errors,  the  random  errors  are  cause  for  m.ore  concern.  As  the 
example  illustrates,  consistent  but  inaccurate  performance  on  the 
part  of  the  operator  can  usually  be  modl:^ied  by  training  or  redesign 
of  equipment,  while  inconsistent  performance  Is  much  harder  to 
correct . 

By  assuming  that  the  bulk  of  the  random  errors  in  performance 
are  the  result  of  the  operator,  a  measure  of  precision  can  be  ob¬ 
tained  by  finding  the  least  square  fit  to  the  data  that  he  produces. 
Operator  precision  can  be  computed  under  the  different  conditions 
of  the  experlmont  by  finding  the  spread  of  results  about  the 
statistically  obtained  criterion. 

Accuracy  measures  can  be  divided  into  two  groups:  acf’uracy 
of  fact  and  adequacy  of  Judgment, 

7.3»3»1  Accuracy  of  Fact 

Included  in  bpls  category  are  these  responses:  appearance 
of  new  targets,  fades  and  reappearances,  IFF  signals, 
initial  course,  change  in  course,  initial  altitude,  change 
in  altitude,  loss  of  target,  initial  vector  to  CAP,  turn 
to  CAP,  homing  instructions  to  CAP,  all  position  reports 
on  targets,  subsequent  vector  to  CAP,  As  can  be  seen  from 
this  list,  the  reports  can  be  compared  directly  to  actual 
occurrence,  v/hlch  ray  in  some  cases  have  to  be  estimated. 

7.3»3»2  Adequacy  of  Judgments 

Into  those  categories  fall  those  kinds  of  actions  which 
can  be  Judged  only  in  terms  of  their  operational  adeouacy. 
This  would  include  the  determination  of  proper  priority 
or.  composition  of  raids,  reporting  of  targets,  tactical 
pvaluatjons,  changes  in  the  AEW  plane's  course,  and 
commitment  of  CAP  ■’"o  intercept. 

The  stimulus  does  not  truly  represent  an  operational 
situation,  and  so  it  is  necessary  to  develop  some  "rules 
of  the  game."  For  the  composition  and  nature  of  a  raid, 
there  are  certain  practices  vliich  are  followed  in  the 
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slnul£jtjon  of  pro’ lens  v'hl^'h  InrUpote  certain  kinds  of 
v'e£.pons.  A- co’^ncrlson  of  the  crope  situation  v;ltb  those 
"rules  of  the  pt.re"  should  provide  the  operator  sone 
levert  po  to  judpe  conposjtlon  and  tjjotlofil  situation. 

Scallnr  teohnlques  otn  be  then  used  t©  rate’  such  reuorts. 

The  opinions  of  persons  v-ho  tre  skilled  In  operational 
judpnents  can  te  systenatically  co’^'Vined  to  evaluate 
these  Judgnents. 

7.^  Internal  T^ee^ures 

The  output  measures  given  In  Section  7«3  ^an  also  he  used 
internally.  It  is  possible  to  got  production  rate,  latency,  accuracy 
neorures  of  each  ACO  as  \7ell  as  the  same  measures  for  the  status 
board.  A  comparison  of  these  values  ’”ith  the  output  will  indicate, 
for  instance,  the  loss  at  the  status  board. 

It  may  veil  bo  that  certain  forms  of  the  system  will  accumulate 
much  more  Information  at  the  status  board  than  ever  becom.es  a  part 
of  the  output  of  the  system. 

A  principle  of  systems  r'Searcb  Is  for  the  experimenter  to 
locate  sources  of  vrrlatlon  in  output  by  description  of  the  Internal 
activity  of  the  system.  In  this  vivy ,  if  the  variables  of  the  ex¬ 
perimental  design  CO  not  account  for  the  variation  in  output,  it  is 
possible  to  check  -further  to  try  to  relate  these  dlffcren^'es  to 
internal  activities.  These  actl'''itles  belong  to  two  major  gioups; 
the  activity  on  the  Inter-communlca tlon  system  and  the  actual  motor 
activity  of  the  orew. 

7.4,1  Activity  on  Intcr-Communl ca tlon  System  (ICS): 

Pcrticular  patterns  of  communlcf tlon  can  be  related  to 
system  output  to  see  mhr-thrr  certain  procedures  contribute  to  a 
more  ef'f'ectlve  performance. 

By  gathering  data  os  to  the  number  of  communlcf tlons  passing 
between  the  various  crew  members,  together  with  distinguishing  the 
initiator  of  the  communication  and  analyzing  the  content  of  these 


nesssf^es,  srversl  results  i^ry  "be  £irti''ip&ted .  First  it  is 
possible  to  check  on  the  formatjon  of  preferred  patterns  betveen 
Individucils .  ITeny  sociological  studies  have  explored  the 
phenomenon  of  how  team  memi  ers  organized  themselves  for  effective 
communication. 

If  social  patterns  affect  system  performance,  they  should 
be  located  and  studied.  It  should  also  be  possible  to  discover 
the  accuracy  and  appropriateness  of  the  communication  gear  itself. 

By  finding  which  channels  have  the  higher  loadings  and  which  are 
least  freauently  used,  it  should  be  possible  to  determine  rhat 
far'ilitles  are  required. 

Another  measure  of  communication  activity  would  be  "cycle 
time."  Under  any  communication  pattern  where  a  group  must  take 
turns  sending  Information  over  a  common  channel,  it  would  be  of 
interest  to  find  how  long  a  time  elapsed  between  successive  uses 
of  the  channel  by  the  same  lndl''^ldual.  There  may  be  a  characterlstl 
rate  inherent  in  a  particular  kind  of  cycle  tha^  could  be  distin¬ 
guished.  This  Information  would  be  of  value  when  the  experimenter 
tries  to  interpret  differences  in  system  performance  under  dif¬ 
ferent  operating  conditions, 

7.4.2  Uotor  Activity  of  the  Crew: 

Into  this  rather  broad  category  fall  time  and  motion 
strdies  of  the  CIC  officer  and  other  key  people  in  the  team.  An 
effort  should  be  made  to  find  v'h?  t  sources  of  Information  these 
people  use  most  frequently.  In  addition  to  disf'overing  the  proper 
placement  of  thrse  displays,  studying  the  spontaneous  crew  behavior 
of  this  kind  under  stress  as  it  relates  to  the  performance  measures 
should  help  to  discover  which  tyne  of  display  is  the  more  useful. 


Det&lled  studies  of  the  potor  aoti’''lty  of  the  Individual 
crew  nenters  sui^h  as  a  micronotlon  study  of  the  Air  Control 
Officers'  use  of  the  controls  on  his  console  would  probably  be 
pore  indicative  for  the  design  of  conponents  than  for  the  system 
design,  but  may  have  some  value  for  understanding  relations  between 
oppra tors. 
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8.0  SYSTEM  PERFORMANCE  DURING  PRELIFINARY  INVESTIGATIONS 

Previous  sections  have  shown  that  the  experimenter  simply 
observes  during  preliminary  investigations.  He  tries  to  pet  a 
general  idea  of  how  the  system  works.  The  principal  results  of 
introductory  studies  of  the  airborne  CIO  have  already  been  dis¬ 
cussed  in  Sections  5«0,  6.0,  and  7»0  which  cover  the  stimulus 
complex,  the  system  variables  and  measurement.  In  addition, 
these  investigations  served  the  purpose  of  training  the  crew  in  the 
use  of  new  CIC  equipment  and  orienting  the  researchers  to  their 
experimental  facilities.  The  information  collected  v/as  primarily 
subjective;  however,  in  the  latter  phases  of  the  preliminary  work, 
it  was  possible  to  gather  some  objective  data. 

There  were  three  distinct  phases  to  the  preliminary  investi¬ 
gations.  The  first  22  practice  sessions,  which  were  run  between 

7  June  and  27  September,  1950,  comprise  Operation  "Art."  During  "Art" 
the  project's  electronic  engineers  were  completing  the  Installation 
of  the  laboratory  airborne  CIC. 

Operation  "Bold"  was  run  between  29  September  and  26  October^, 
1950;  it  includes  practice  sessions  23  through  35* 

The  nine  experlm.ental  sessions  conducted  during  the  period 

8  to  15  January  1951?  make  up  Operation  "Candy."  "Candy"  was. 

In  fact,  the  preliminary  run  to  the  first  exploratory  experiment, 
which  will,  of  course,  be  reported  subsequently  in  a  technical 
report  in  this  series. 

8.1  Description  of  Operational  Conditions 

Throughout  Art,  Bold,  and  Candy  an  effort  was  made  to  explore 
different  ways  of  operating  the  system.  During  this  survey,  a 
record  was  kept  of  the  operating  conditions  used,  the  nature  of  the 
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stimulus  presented,  and  the  kinds  of  data  collected.  Tables  lA, 

IB,  and  1C  summarize  these  facts  for  the  three  operations.  This 
information  Is  reported  in  detail  to  illustrate  the  range  of 
conditions  explored  in  the  preliminary  work, 

8,2  Results  of  Operation  Bold 

The  refinement  of  experimental  controls  and  the  training  of 
both  the  crew  and  experimenters  during  Operation  Art  permitted  the 
collection  of  some  objective  data  during  Bold.  However,  even  the 
objective  data  obtained  during  Bold  and  Candy  were  collected  under 
loosely  controlled  experimental  conditions.  These  data  will  serve 
to  indicate  only  the  general  range  into  which  system  performance 
falls.  It  will  be  possible  to  infer,  for  instance,  that  the  number 
of  reports  per  minute  will  be  in  the  range  of  two  to  eight  rather  than 
in  the  range  of  20  to  30.  The  presentation  of  these  results  is  in 
order  because  they  are  the  first  indication  of  what  an  airborne  CIC 
can  do.  These  performance  records  are,  as  a  matter  of  fact,  the 
first  gathered  on  the  operation  of  any  CIC  in  a  laboratory  situation. 
In  reporting  the  results,  the  outline  of  measures  described 
in  Paragraph  7.3  will  be  followed.  Only  production  measures  will 
be  reported  for  Operation  Bold,  however;  instrumentation  had  not 
been  developed  at  that  time  to  obtain  latency  and  accuracy  measures. 

8.2.1  Production  During  Operation  Bold:  Two  production  measures 
can  be  reported  for  the  system  performance  during  Bold. 

8. 2. 1.1  Number  of  Reports  per  Minute 

On  the  average  the  airborne  CIC  produced  slightly  less 
than  five  reports  per  minute.  This  measure  includes  all  types 
of  reports  and  is  a  gross  indicator  of  the  amount  of  work  done  by 
the  system.  In  Figure  3  is  shown  the  average  number  of  reports 
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Description  of  Conditions 
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made,  minute  by  minute,  durlnp  Bold.  Both  the  actual  data,  and  a 
"smoothed”  curve  are  shown.  There  Is  irregularity  in  production 
from  minute  to  minute,  and  no  pattern  emerges.  In  the  first  ten 
minutes  of  any  session  the  stimulus  load  was  being  built  up,  ac¬ 
counting  for  the  low  production  at  that  time. 

8. 2. 1.2  Number  of  Reports  per  Target  per  Minute 

Actually,  the  inverse  of  the  number  of  reports  per  target 
per  minute  is  used  for  convenience  in  computation;  that  Is,  the  time 
between  successive  reports  on  the  same  target.  Table  2  shov’s  the 
average  time  between  successive  reports  on  the  same  target  for  nine 
of  the  13  sessions  of  Bold;  the  results  from  the  remaining  sessions 
could  not  be  used  because  of  the  manner  in  which  performance  was 
recorded.  For  example,  in  Practice  Session  23  an  average  time 
between  successive  reports  was  found  for  each  of  12  targets.  A  mean 
of  2.71  minutes  and  a  standard  deviation  of  1.44  minutes  for  these 
12  values  was  then  found.  Table  2  shows  the  number  of  targets  upon 
which  the  reported  mean  and  standard  deviation  for  each  session  was 
based.  This  means  that  if  a  position  report  is  given  on  Target  7 
at  time  10,  another  position  report  on  that  target  will  not  be 

forthcoming  until  almost  Time  13o 

TABLE  2 


AVERAGE  TIME  BETWEEN  SUCCESSIVE  REPORTS  ON  SAME 
TARGET  (BOLD)  (IN  MINUTES) 


Practice 

Session 

Mean  Time 
Between 
Successive 
Reports 

Number  of 
Targets 
Used 

Standard  Deviation  of 
Values  for  Each  Target 
About  Mean  Value  for 
Session 
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1.44 

24 
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TIME  IN  MINUTES 

FI0URE3.  AVERAGE  NUMBER  OF  REPORTS,  MINUTE  BY 
MINUTE  FOR  OPERATION  BOLD. 
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The  averapp  value  for  the  nine  neens  In  Table  2  Is  3.48 
ninutes;  the  standard  deviation  Is  1.31  minutes.  The  average  time 
between  succeeding  reports  is  a  more  refined  measure  than  the  number 
of  reports/ninute  shown  in  Figure  3.  The  consistency  with  which 
the  targets  in  any  stimulus  ere  handled  is  shovn  by  the  standard 
deviation  in  Cession  35?  for  eyar^ple,  v»here  the  standard  deviation 
is  .58  minute.  The  system  is  reporting  on  one  of  the  11  targets 
as  often  as  it  renorts  on  another.  In  Session  25)  however,  where 
the  standard  deflation  is  3*30,  the  system  is  not  producing  the 
same  number  of  reports  on  one  target  as  consistently  as  on  another, 
"^ho  data  also  indicate  that  there  is  considerable  improvement  in  the 
system's  production  throughout  Bold.  There  is  less  lag  between 
successive  reports  as  the  investigations  continued,  possibly  because 
of  the  experience  that  the  crew  was  gaining. 

8.3  Results  of  Operation  Candy 

Candy  was  a  preliminary  run  of  a  system  study  designed  to 
compare  the  ef ‘’’ect iveness  of  four  different  operating  procedures. 
These  procedures  are  designated  Ant,  Bat,  Card,  or  Dove.  Further 
identification  of  the  operating  conditions  is  not  necessary  because 
valid  comparisons  cannot  be  made  on  the  basis  of  these  data.  Candy 
served  its  purpose  by  clarifying  the  issues  which  were  then  subjected 
to  experimental  study  in  Operation  Dandy,  to  be  reported  later. 

It  is  possible  to  report  more  extensive  results  of  Candy  in 
all  three  categories  of  measurement:  production,  latency  and 
accuracy. 

8.3.1  Production  Durlnr  Candy:  These  production  measures  will 
be  reported:  number  of  reports  per  minute;  average  time  between 
successive  reports  on  the  same  target,  the  average  time  to  transmit 
a  position  report. 
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8, 3. 1.1  Number  of  Reports  Per  Wlnute 

Table  3  shows  the  number  of  reports  mode  by  the  system 
during  eight  of  the  nine  sessions  (laboratory  difficulties 
occurred  during  the  first  :\in  in  the  series  so  that  those  results 
are  not  given).  Position  reports,  course  and  speed  reports,  number 
of  composition  or  evaluative  reports,  together  with  the  total  number 
of  reports,  are  tabulated  for  each  run. 

TABLE  3 

NW®ER  OF  REPORTS  MDE  BY  THE  SYSTEI^  PER  MINUTE 

Condition  Condition  Condition  Condition 
Ant  Bat  Card  Dove 

Kind  of  Report  Run  7  Run  2  Run  5  Run  6  Run  ^  Run  4  Run  9 


Position 

3.7 

3.2 

3.0 

3.2 

4.0 

3.5 

4.1  4.7 

Course  and 

Speed  .5 

.5 

.4 

.4 

.3 

.4  . 5 

Eva luative 

.5 

.2 

o2 

.4 

.2 

.3 

.2  .3 

Total 

4.6 

4.0 

3.7 

4.1 

4.7 

4ol 

4.8  5.4 

It  will  be 

noted  that 

the  data 

in 

this  table  co’^pares 

to  that 

contained 

in  Figure  3* 

About 

83^  of  the 

total 

reports 

are  position 

reports,  another  10^  of  the  total  are  course  and  speed  reports,  while 
the  remaining  6^  are  evaluative  reports. 

8. 3.1.2  Average  Time  Between  Successive  Reports  on  Same  Target 

The  same  measure  as  reported  in  Table  2  with  regard  to 

Operation  Bold  was  calculated  for  Candy.  Those  dote  are  contained 

in  Table  4.  In  each  case,  the  mean  and  standard  deviation  is  based 

upon  the  average  values  obtained  for  each  of  five  targets. 

TABLE  4 

AVERAGE  TIME  BETWEET’  SUCCESSIVE  REPORTS  ON  SAT'^E  TARGET 


(GANDY)  (IN  I'^INUTES) 


Statistic 

'  CUTTaiTlbTi — 
_ Ant 

(JonaiTion 

Bat 

— CAnffitldh 
Card 

— Coh'dlTlbh 
Dove 

Run  7  Run  2 

Run  5  Run  6 

Run  3  Ri’n  8 

Run  4  Run  9 

Mean  Time  Between 
Successive  Reports 
Standard  Deviation 

4.74  2.66 
1.29  .12 

2.47  1.90 

.51  .55 

2.13  2.57 
.21  .30 

2.35  2.12 
.47  .16 

Systen  performance  during  Cendy  seems  to  be  better  than  that 
during  Bold.  This  is,  of  course,  to  be  expected.  Both  the 
means  and  standard  deviations  ore  lov.'er  than  in  Table  2,  which 
indicates  improvement. 

8. 3.1*3  Average  Time  to  Transmit  Position  Report 

This  measure  is  one  indicator  of  transmission  rate.  There 
are  many  difficulties  in  trying  to  get  a  precise  measure  of  verbal 
activities  over  su<^h  a  short  period  of  time.  The  values  for  the 
mean  time  to  transmit  one  position  report,  the  standard  deviation 
and  the  number  of  measures  upon  which  those  statistics  were  based 
are  tabulated  for  eight  runs  in  Table 


TABLE 

AVERAGE  TIME  TO  TRANSMIT  A  POSITION  REPORT 

(IN  SECONDS) 


Statistic 

Condition 

Ant 

Condition 

Bat 

Condition 

Card 

Condition 

Dove 

EinxlISS9sfl 

Run  4  Run  9 

No.  of  Observations 
on  which  statistic  . 

is  based  « 

/verage  Time  J  «« 

Standard  Deviation  2.26  1.99 

17  7 

5.5  6.3 

2.85  ♦ 

80  33 

6.8  8.5 

2.11  3*26 

28  23 

7.0  8.0 

1.94  2.93 

*N  too  small  to  compute  meaningful  standard  deviation 
8.3.2  Latency  Measures  During  Candy;  These  latency  measures  will 
be  reported:  latency  in  detef^tion  of  new  bogeys,  and  latency  in 
detecting  courses  and  speeds  on  new  targets. 

8. 3.2.1  Latency  in  Detecting  New  Targets 

Table  6  shows  how  long  it  takes  the  system  to  detect  a 
new  target.  The  mean  and  standard  deviations  of  this  detection 
time  are  based,  in  each  case,  upon  ten  detections. 


■TABLE  6 


TII'TE  NECESSARY  FOR  THE  SYSTET^  TO  DETECT  A  NEV’  TARGET 

(IN  HINUTES) 


There  3s  conrlderable  variance  in  system  performance  so  far 
as  this  measure  is  concerned,  ^liore  the  sign  ">"  3s  used,  the 
average  time  is  somev/hat  greater  than  the  value  quoted;  this  is 
occasioned  by  the  fact  that  the  system  did  not  produce  course  and 
speed  on  one  of  the  four  targets  during  the  tine  that  recordings 
v/ere  made.  Some  procedure  is  lrdic£.ted  to  insure  that  courses  and 
speeds  rn  all  targets  are  produced. 
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8.3*3  Accuracy  I'^easures  during  Candvs  The  following  accuracy 
ncasures  will  be  reported:  accuracy  of  position  reports  and 
accuracy  of  course  reports. 

8. 3*3*1  Accuracy  of  Position  Reports 

This  measure  was  obtained  by  first  plotting,  for  each 
run,  the  position  reports  on  five  targets.  Because  each  of 
these  targets  proceeded  in  a  straight  line,  it  was  possible  to 
lay  a  transparent  strip  along  the  plotted  positions  for  any  one 
target.  This  strip  was  cut  to  be  five  miles  wide  according  to 
the  scale  of  the  plot;  it  v/as  put  down  to  include  as  many  points 
under  it  as  possible.  The  reported  positions  lying  outside  this 
range  were  counted  as  bad  reports.  Actually,  under  operational 
conditions,  a  track  determined  by  positions  falling  within  a  ten^ 
mile  strip,  for  instance,  might  still  be  considered  ’’good." 

The  five-mile  limit  was  set  arbitrarily  in  order  to  get  an 
indication  of  the  per  cent  of  *’bad"  reports;  these  are  contained 
in  Table  8. 


TABLE  8 

PER  CENT  OF  BAD  POSITION  REPORTS 


Condition 

Ant 

Condition 

Bat 

Condition 

Card 

Condition 

Dove 

Run  7  Run  2 

Run  5  Run  6 

Run  3  Run  b 

Run  4  Run  9 

Per  cent 
position 

of  "bad" 
reports 

27  35 

28  24 

32  21 

30  31 

There  is  considerable  question  about  the  appropriateness  of  this 
measure.  Different  techniques  for  des<^ribing  the  accuracy  of 
position  reports  will  be  explored  in  future  studieso 


8.3»3«2  Accuracy  of  Course  Reports 

In  order  to  pet  an  estl’^'ate  of  the  accuracy  of  course 
reports,  It  was  necessary  to  find  a  criterion  apainst  vrhich  to 
compare  the  reported  course.  This  was  done  by  recording  the 
course  represented  by  the  transparent  strip  which  was  placed  to 
find  the  number  of  "bad”  reports.  The  criterion  course  was  an 
average  fit  to  the  position  reports  produced.  The  differences 
between  the  reported  courses  and  the  criterion  courses  were  tabu¬ 
lated  for  five  targets  in  each  run.  The  average  errors  are  tabu¬ 
lated  in  Table  9. 


TABLE  9 

COURfE  ERRORS 
(IP  DEGREES) 


Condition 

Condition 

Condition 

Condition 

Ant 

Bat 

Ca  rd 

Dove 

Run  7  Run  2 

Run  5  Run  6 

Run  3  Run  8 

Run  4  Run  9 

Course  of  Error  10  (1) 

42(2)  16 

13 

17  43(2) 

(1)  Errors  not  reported  because  of  transcription  dif¬ 
ficulties. 

(2)  Large  average  error  caused  by  a  report  on  one  target, 

In  each  cere,  being  180  degrees  v;rong,  or  a  gross  error. 


